aleksis/AlekSIS-App-Alsijil
Watch
10
0
Fork
You've already forked AlekSIS-App-Alsijil
 0
Code Issues 48 Pull requests 7 Projects Releases 3 Packages 1 Activity Actions

Resolve "Add task for checking plausibility of data" #482

Merged
hansegucker merged 30 commits from 91-add-task-for-checking-plausibility-of-data into master 2021-01-08 13:01:14 +01:00
Conversation 45 Commits 30 Files changed 2 +222
hansegucker commented 2020-09-19 12:06:54 +02:00
Owner
Copy link

Closes #91

Closes #91
hansegucker commented 2020-09-19 18:53:52 +02:00
Author
Owner
Copy link

added 1 commit

  • 07106d1b - Init data check system and add first check for personal notes

Compare with previous version

added 1 commit <ul><li>07106d1b - Init data check system and add first check for personal notes</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3353&start_sha=5833dfa2893c0e63b1b35ebe27b580409c29dc90)
hansegucker commented 2020-09-19 19:05:14 +02:00
Author
Owner
Copy link

added 1 commit

  • 60b52a14 - Move data check to own (optional) celery task

Compare with previous version

added 1 commit <ul><li>60b52a14 - Move data check to own (optional) celery task</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3354&start_sha=07106d1b0819a8fcd8fd6cd4c69c657b33054282)
hansegucker commented 2020-09-20 17:02:54 +02:00
Author
Owner
Copy link

added 3 commits

  • 7e9c9b66 - Add missing migration
  • 52883be8 - Send emails to notify about data problems
  • d49f14ee - Merge remote-tracking branch...

Compare with previous version

added 3 commits <ul><li>7e9c9b66 - Add missing migration</li><li>52883be8 - Send emails to notify about data problems</li><li>d49f14ee - Merge remote-tracking branch...</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3355&start_sha=60b52a141231ab13845b9895095da1eca50be665)
hansegucker commented 2020-09-20 17:09:03 +02:00
Author
Owner
Copy link

added 1 commit

  • b34fad00 - Update menu representation for data checks

Compare with previous version

added 1 commit <ul><li>b34fad00 - Update menu representation for data checks</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3356&start_sha=d49f14eef895a9a4fb3de8a1e50e650edc30dc2e)
tteichler commented 2020-09-22 17:07:44 +02:00
Owner
Copy link

Maybe you also can add a custom health check to the system status page.

https://github.com/KristianOellegaard/django-health-check#writing-a-custom-health-check

Maybe you also can add a custom health check to the system status page. https://github.com/KristianOellegaard/django-health-check#writing-a-custom-health-check
nik commented 2020-09-25 16:36:40 +02:00
Owner
Copy link

Please evaluate django-datawatch.

Please evaluate django-datawatch.
nik commented 2020-09-25 16:36:44 +02:00
Owner
Copy link

https://lmgtfy.com/?q=python+docstring

https://lmgtfy.com/?q=python+docstring
nik commented 2020-09-25 16:36:45 +02:00
Owner
Copy link

Please use class-based views.

Please use class-based views.
hansegucker commented 2020-11-14 16:58:53 +01:00
Author
Owner
Copy link

added 196 commits

  • b34fad00...4a91b607 - 195 commits from branch master
  • 6f41b16b - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data

Compare with previous version

added 196 commits <ul><li>b34fad00...4a91b607 - 195 commits from branch <code>master</code></li><li>6f41b16b - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3744&start_sha=b34fad00dff324ffea6efdc3f75bfb38a89ecfbb)
hansegucker commented 2020-11-14 17:15:55 +01:00
Author
Owner
Copy link

changed this line in version 6 of the diff

changed this line in [version 6 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3763&start_sha=6f41b16bd6b9977e208cf7ac67d55853d34cebcb#15793436ffa21b8e1a3c3610e43f80aed3c1d7a8_15_9)
hansegucker commented 2020-11-14 17:15:56 +01:00
Author
Owner
Copy link

changed this line in version 6 of the diff

changed this line in [version 6 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3763&start_sha=6f41b16bd6b9977e208cf7ac67d55853d34cebcb#17fa7705c025f0876e0fe2c72d23ddad96fcf6f2_854_842)
hansegucker commented 2020-11-14 17:15:57 +01:00
Author
Owner
Copy link

added 1 commit

  • 27413aff - Move base data check system to core

Compare with previous version

added 1 commit <ul><li>27413aff - Move base data check system to core</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3763&start_sha=6f41b16bd6b9977e208cf7ac67d55853d34cebcb)
hansegucker commented 2020-11-14 17:18:51 +01:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>2df54418 - Also move template to core</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3765&start_sha=27413affba971d674171477d0439cfbccf9b04c8)
hansegucker commented 2020-11-14 18:09:34 +01:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>9838f025 - Fix imports in data checks</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3768&start_sha=2df544180e0aa733289df6dbc3cd61bebeffbfe3)
hansegucker commented 2020-11-14 18:11:46 +01:00
Author
Owner
Copy link

Blocked by AlekSIS/official/AlekSIS-Core!779

Blocked by AlekSIS/official/AlekSIS-Core!779
hansegucker commented 2020-11-14 18:25:45 +01:00
Author
Owner
Copy link

added 1 commit

  • e8129856 - Move email templates and preferences for data checks to core

Compare with previous version

added 1 commit <ul><li>e8129856 - Move email templates and preferences for data checks to core</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3772&start_sha=9838f025bfbb166378d1d4d0d4980695dadf86c0)
hansegucker commented 2020-11-17 20:06:34 +01:00
Author
Owner
Copy link

added 18 commits

  • e8129856...405aa743 - 15 commits from branch master
  • e1a7b126 - Write data check NoGroupsOfPersonsSetInPersonalNotesDataCheck
  • 9e0323c2 - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data
  • ebff52ea - Reformat

Compare with previous version

added 18 commits <ul><li>e8129856...405aa743 - 15 commits from branch <code>master</code></li><li>e1a7b126 - Write data check NoGroupsOfPersonsSetInPersonalNotesDataCheck</li><li>9e0323c2 - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li><li>ebff52ea - Reformat</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3870&start_sha=e8129856046318952640567342e9bc0d66cfc816)
hansegucker commented 2020-11-18 17:50:40 +01:00
Author
Owner
Copy link

added 1 commit

  • ba6eeed0 - Add data check for lesson documentations on holidays

Compare with previous version

added 1 commit <ul><li>ba6eeed0 - Add data check for lesson documentations on holidays</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3876&start_sha=ebff52ea7c2b8bf0ead9f7aeea4e892be68bcb8c)
hansegucker commented 2020-11-18 17:52:51 +01:00
Author
Owner
Copy link

Maybe someone find a more efficient way to do this stuff.

Maybe someone find a more efficient way to do this stuff.
hansegucker commented 2020-11-19 21:11:38 +01:00
Author
Owner
Copy link

added 5 commits

  • ba6eeed0...abfdb175 - 3 commits from branch master
  • 0d16d3cb - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data
  • 7606d5e2 - Fix lint issues

Compare with previous version

added 5 commits <ul><li>ba6eeed0...abfdb175 - 3 commits from branch <code>master</code></li><li>0d16d3cb - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li><li>7606d5e2 - Fix lint issues</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3886&start_sha=ba6eeed01a92de66250804e0d9a9c1703d4c9c61)
hansegucker commented 2020-11-20 16:14:57 +01:00
Author
Owner
Copy link

added 2 commits

  • 3cc04ff9 - 1 commit from branch master
  • fa968769 - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data

Compare with previous version

added 2 commits <ul><li>3cc04ff9 - 1 commit from branch <code>master</code></li><li>fa968769 - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3887&start_sha=7606d5e22059c9bbd74e3e5399f2f2c37004cd64)
hansegucker commented 2020-11-29 11:22:42 +01:00
Author
Owner
Copy link

added 6 commits

  • fa968769...5fb87b07 - 5 commits from branch master
  • 335b5027 - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data

Compare with previous version

added 6 commits <ul><li>fa968769...5fb87b07 - 5 commits from branch <code>master</code></li><li>335b5027 - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=3973&start_sha=fa968769584904ec1f96cf670c28734763591911)
hansegucker commented 2020-12-26 18:01:15 +01:00
Author
Owner
Copy link

added 14 commits

  • 335b5027...053be75f - 13 commits from branch master
  • 469bbd80 - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data

Compare with previous version

added 14 commits <ul><li>335b5027...053be75f - 13 commits from branch <code>master</code></li><li>469bbd80 - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4075&start_sha=335b5027d05ff4a41563133d38cd4d1efcff9cda)
hansegucker commented 2020-12-26 18:10:31 +01:00
Author
Owner
Copy link

added 1 commit

  • 25067f6e - Add check for personal notes on holidays

Compare with previous version

added 1 commit <ul><li>25067f6e - Add check for personal notes on holidays</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4076&start_sha=469bbd80adce1c311fde51246c5f9624667ad8c6)
hansegucker commented 2020-12-26 19:38:59 +01:00
Author
Owner
Copy link

added 3 commits

  • b2f2c7d2 - Add data check to ensure that there are no excuses without absences
  • f42357a6 - Remove superfluous query from data checks
  • 3c99004a - Add missing reset solve option for data checks

Compare with previous version

added 3 commits <ul><li>b2f2c7d2 - Add data check to ensure that there are no excuses without absences</li><li>f42357a6 - Remove superfluous query from data checks</li><li>3c99004a - Add missing reset solve option for data checks</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4079&start_sha=25067f6e13ffd74ef6bdf328fad7cd762bfe977e)
hansegucker commented 2020-12-26 19:44:56 +01:00
Author
Owner
Copy link

changed this line in version 18 of the diff

changed this line in [version 18 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4082&start_sha=3c99004ae87a9f5b6a3f1eba6627fc1293de7dab#15793436ffa21b8e1a3c3610e43f80aed3c1d7a8_142_126)
hansegucker commented 2020-12-26 19:44:59 +01:00
Author
Owner
Copy link

added 1 commit

  • bd0ae76f - Simplify registering of data check results by adding a dedicated method

Compare with previous version

added 1 commit <ul><li>bd0ae76f - Simplify registering of data check results by adding a dedicated method</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4082&start_sha=3c99004ae87a9f5b6a3f1eba6627fc1293de7dab)
hansegucker commented 2020-12-26 19:46:07 +01:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>d1d30c65 - Fix lint</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4084&start_sha=bd0ae76f7562554782363e856de10afd3c8ba5a0)
hansegucker commented 2020-12-26 20:50:37 +01:00
Author
Owner
Copy link

assigned to @nik

assigned to @nik
hansegucker commented 2020-12-26 20:50:43 +01:00
Author
Owner
Copy link

requested review from @debdolph

requested review from @debdolph
hansegucker commented 2020-12-26 20:50:50 +01:00
Author
Owner
Copy link

removed review request for @debdolph

removed review request for @debdolph
hansegucker commented 2020-12-26 20:52:06 +01:00
Author
Owner
Copy link

marked this merge request as ready

marked this merge request as **ready**
hansegucker commented 2020-12-26 20:52:33 +01:00
Author
Owner
Copy link

@nik Any idea?

@nik Any idea?
hansegucker commented 2020-12-28 11:56:50 +01:00
Author
Owner
Copy link

added 1 commit

  • 96c09d5a - Load data checks from related models instead of using a custom registry

Compare with previous version

added 1 commit <ul><li>96c09d5a - Load data checks from related models instead of using a custom registry</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4106&start_sha=d1d30c65fece69393529fa573831a11e3d7c208d)
nik commented 2020-12-28 22:38:42 +01:00
Owner
Copy link

What we basically want is something like this SQL:

select doc.* from doc, holidays where holidays.start <= to_date(concat('2020', week), 'IYYYIWW') + weekday - 1 and to_date(concat('2020', week), 'IYYYIWW') + weekday - 1 <= holidays."end";

I got as far as translating the date conversion:

weekday_to_date = Func(Concat("2020", F("week")), "IYYYIW", function="TO_DATE") + F("weekday") - Value(1

Now, how do we do a JOIN without a foreign key relation in Django?

If only we had SQLAlchemy ☹

What we basically want is something like this SQL: ```sql select doc.* from doc, holidays where holidays.start <= to_date(concat('2020', week), 'IYYYIWW') + weekday - 1 and to_date(concat('2020', week), 'IYYYIWW') + weekday - 1 <= holidays."end"; ``` I got as far as translating the date conversion: ```python weekday_to_date = Func(Concat("2020", F("week")), "IYYYIW", function="TO_DATE") + F("weekday") - Value(1 ``` Now, how do we do a `JOIN` without a foreign key relation in Django? If only we had SQLAlchemy ☹
nik commented 2020-12-28 22:47:48 +01:00
Owner
Copy link

In any case, this field expression up there might be (part of) the solution for AlekSIS-App-Chronos#81

In any case, this field expression up there might be (part of) the solution for AlekSIS-App-Chronos#81
hansegucker commented 2021-01-04 16:06:32 +01:00
Author
Owner
Copy link

The actual working variant 😉 :

weekday_to_date = ExpressionWrapper(Func(Concat(F("year"), F("week")), Value("IYYYIW"), output_field=DateField(), function="TO_DATE") + F("lesson_period__period__weekday"), output_field=DateField())
The actual working variant :wink: : ```python weekday_to_date = ExpressionWrapper(Func(Concat(F("year"), F("week")), Value("IYYYIW"), output_field=DateField(), function="TO_DATE") + F("lesson_period__period__weekday"), output_field=DateField()) ```
hansegucker commented 2021-01-04 16:12:24 +01:00
Author
Owner
Copy link

resolved all threads

resolved all threads
hansegucker commented 2021-01-04 16:12:25 +01:00
Author
Owner
Copy link

added 1 commit

  • e208d546 - Make holiday data checks much more efficient by using SQL functions

Compare with previous version

added 1 commit <ul><li>e208d546 - Make holiday data checks much more efficient by using SQL functions</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4160&start_sha=96c09d5a495f425ddee08fcab52120761684c681)
hansegucker commented 2021-01-04 16:52:31 +01:00
Author
Owner
Copy link

added 5 commits

  • e208d546...7990af6c - 4 commits from branch master
  • eeb5f521 - Merge branch 'master' into 91-add-task-for-checking-plausibility-of-data

Compare with previous version

added 5 commits <ul><li>e208d546...7990af6c - 4 commits from branch <code>master</code></li><li>eeb5f521 - Merge branch &#39;master&#39; into 91-add-task-for-checking-plausibility-of-data</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4162&start_sha=e208d5469ebd4b9ce3f2f6d3d3d34cb117dfc9a4)
nik commented 2021-01-06 20:05:48 +01:00
Owner
Copy link

This should not be a nested loop. Build a query containing all holiday dates and then check them all in one batch.

This should not be a nested loop. Build a query containing all holiday dates and then check them all in one batch.
hansegucker commented 2021-01-07 19:49:13 +01:00
Author
Owner
Copy link

changed this line in version 23 of the diff

changed this line in [version 23 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4176&start_sha=eeb5f5214274ab292ca0c60753fecbab67d2619e#15793436ffa21b8e1a3c3610e43f80aed3c1d7a8_130_128)
hansegucker commented 2021-01-07 19:49:16 +01:00
Author
Owner
Copy link

added 1 commit

  • 1b44092a - [Data checks] Combine holiday queries to one big query

Compare with previous version

added 1 commit <ul><li>1b44092a - [Data checks] Combine holiday queries to one big query</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/92/diffs?diff_id=4176&start_sha=eeb5f5214274ab292ca0c60753fecbab67d2619e)
hansegucker commented 2021-01-07 19:49:51 +01:00
Author
Owner
Copy link

resolved all threads

resolved all threads
nik commented 2021-01-08 13:01:14 +01:00
Owner
Copy link

mentioned in commit 891584bf92

mentioned in commit 891584bf92f9b856cae0cd03f548b6b59c35bd54
nik merged commit 891584bf92 into master 2021-01-08 13:01:14 +01:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
Security

   
TeX
Needs LaTeX template

  
auto-update
  
board
done

  
board
ready

  
board
todo

  
check
delete-eslint-rc-js

  
check
update-builddeps-package-json

  
check
update-eslint-rc-js

  
check
update-gitignore

  
check
update-merge-request-template

  
check
update-prettier-ignore

  
check
update-pyproject-toml

  
check
update-renovate-json

  
check
update-tox-ini

  
part
backend
 Backend (view code, models, logic, etc.)

  
part
ci

  
part
docs
 Issue or addition to the documentation

  
part
frontend
 Frontend (templates, etc.)

  
part
i18n
 Translation or internationalisation issue

  
part
non-technical
 Non-technical (policy, maintenance, etc.) issue or discussion

  
part
packaging
 Issue concerning the packaging of AlekSIS and components

  
prio
1
 Priority High

  
prio
2
 Priority Medium

  
prio
3
 Priority Low

  
release-mr-5.x
  
size
large

  
size
medium

  
size
small

  
source
customer
 Requested by a customer or affiliated user

  
source
customer::fsmw

  
source
customer::fss

  
source
customer::teckids

  
source
downstream
 Issue or merge request from other projects using AlekSIS or parts of it

  
type
breaking

  
type
bug
 Bug

  
type
feature
 New feature

  
type
refactoring

  
workflow
blocked
 Issue is blocked by another issue

  
workflow
confirmed
 The bug is reproducible and will be fixed

  
workflow
current-todo

  
workflow
discussing
 Needs to be discussed before implementation

  
workflow
new-app
 Requires new AlekSIS app, which has to be created

  
workflow
wontfix
 The bug is not reproducible or the feature was decided against

  
source::aleksolotl
No labels
Security
TeX
auto-update
board
done
board
ready
board
todo
check
delete-eslint-rc-js
check
update-builddeps-package-json
check
update-eslint-rc-js
check
update-gitignore
check
update-merge-request-template
check
update-prettier-ignore
check
update-pyproject-toml
check
update-renovate-json
check
update-tox-ini
part
backend
part
ci
part
docs
part
frontend
part
i18n
part
non-technical
part
packaging
prio
1
prio
2
prio
3
release-mr-5.x
size
large
size
medium
size
small
source
customer
source
customer::fsmw
source
customer::fss
source
customer::teckids
source
downstream
type
breaking
type
bug
type
feature
type
refactoring
workflow
blocked
workflow
confirmed
workflow
current-todo
workflow
discussing
workflow
new-app
workflow
wontfix
source::aleksolotl
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
nik
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
aleksis/AlekSIS-App-Alsijil!482
No description provided.