aleksis/AlekSIS-App-Alsijil
Watch
10
0
Fork
You've already forked AlekSIS-App-Alsijil
 0
Code Issues 48 Pull requests 7 Projects Releases 3 Packages 1 Activity Actions

Performance optimizations #480

Merged
hansegucker merged 29 commits from feature/performance-optimizations into master 2020-10-27 17:09:28 +01:00
Conversation 55 Commits 29 Files changed 9 +334 -205
hansegucker commented 2020-09-07 14:28:07 +02:00
Owner
Copy link

Close #107

Close #107
hansegucker commented 2020-09-07 14:28:07 +02:00
Author
Owner
Copy link

changed milestone to %2

changed milestone to %2
hansegucker commented 2020-09-07 14:28:36 +02:00
Author
Owner
Copy link

changed the description

changed the description
hansegucker commented 2020-09-07 15:22:22 +02:00
Author
Owner
Copy link

added 5 commits

  • 6c29d76d...a7727a68 - 2 commits from branch master
  • e9282bc9 - Merge branch 'master' into 73-add-rules-and-permissions
  • 45330d48 - Simplify queryset in RegisterAbsenceForm
  • 78772300 - Merge branch '73-add-rules-and-permissions' into feature/performance-optimizations

Compare with previous version

added 5 commits <ul><li>6c29d76d...a7727a68 - 2 commits from branch <code>master</code></li><li>e9282bc9 - Merge branch &#39;master&#39; into 73-add-rules-and-permissions</li><li>45330d48 - Simplify queryset in RegisterAbsenceForm</li><li>78772300 - Merge branch &#39;73-add-rules-and-permissions&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3250&start_sha=6c29d76d737bae7a58706c034077d482bc16ff3b)
hansegucker commented 2020-09-09 13:09:53 +02:00
Author
Owner
Copy link

added 1 commit

  • ea8239e3 - Reorder permission rules in order to make them more efficient

Compare with previous version

added 1 commit <ul><li>ea8239e3 - Reorder permission rules in order to make them more efficient</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3276&start_sha=78772300a267402f7022e581d9654a5fe9d020aa)
hansegucker commented 2020-09-09 13:32:11 +02:00
Author
Owner
Copy link

added 1 commit

  • f56f70a5 - Optimize has_person_group_object_perm

Compare with previous version

added 1 commit <ul><li>f56f70a5 - Optimize has_person_group_object_perm</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3277&start_sha=ea8239e315f8b2b09bcd6bdc09cc4dd2ad51690e)
hansegucker commented 2020-09-09 14:26:36 +02:00
Author
Owner
Copy link

marked as a Work In Progress

marked as a **Work In Progress**
yuha commented 2020-09-09 15:14:49 +02:00
Member
Copy link

added 1 commit

  • f0cb7404 - Optimize performance of predicates with custom permission queries

Compare with previous version

added 1 commit <ul><li>f0cb7404 - Optimize performance of predicates with custom permission queries</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3282&start_sha=f56f70a58bb2a227266473e1b883651e23920e69)
yuha commented 2020-09-09 15:24:57 +02:00
Member
Copy link

added 1 commit

  • 32ef21a6 - Optimize performance of predicates with custom permission queries

Compare with previous version

added 1 commit <ul><li>32ef21a6 - Optimize performance of predicates with custom permission queries</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3283&start_sha=f0cb7404a5cb2551230e40554b30785d7f3d1dda)
hansegucker commented 2020-09-09 17:29:32 +02:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>24ef1e73 - Simplify queries</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3285&start_sha=32ef21a64f1f7b96a6516f9881ceb7f68940fc76)
hansegucker commented 2020-09-09 17:40:18 +02:00
Author
Owner
Copy link

added 1 commit

  • 50459916 - Rewrite some rules more efficient

Compare with previous version

added 1 commit <ul><li>50459916 - Rewrite some rules more efficient</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3286&start_sha=24ef1e73dce1e0679bed3cf1411f1562c438c566)
hansegucker commented 2020-09-09 17:45:11 +02:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>940687dc - Convert queries to lists</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3287&start_sha=50459916cd489e0dfb6c3bd2577a02c168e470d0)
hansegucker commented 2020-09-10 13:37:41 +02:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>cbe2050b - Optimize model extensions</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3293&start_sha=940687dc9f53ba2f1e88e229acccb75ec0082c22)
hansegucker commented 2020-09-10 17:35:47 +02:00
Author
Owner
Copy link

added 3 commits

Compare with previous version

added 3 commits <ul><li>eb2f42e7 - Prefetch form select data</li><li>707cf04f - Use get_content_type_by_perm in predicates</li><li>3b8503d8 - Reformat</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3294&start_sha=cbe2050b3c28231fae67340766ab244c317c14e0)
hansegucker commented 2020-09-10 17:44:18 +02:00
Author
Owner
Copy link

added 1 commit

Compare with previous version

added 1 commit <ul><li>b62a14d2 - Remove wrong set prefetch</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3295&start_sha=3b8503d8411fd56ed41e783195025dbf47ba58be)
hansegucker commented 2020-09-10 18:23:05 +02:00
Author
Owner
Copy link

added 1 commit

  • 66071535 - Filter prefetched personal notes

Compare with previous version

added 1 commit <ul><li>66071535 - Filter prefetched personal notes</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3296&start_sha=b62a14d219bf3af7086fde0dc0c176fc283d7994)
hansegucker commented 2020-09-10 18:32:56 +02:00
Author
Owner
Copy link

added 2 commits

Compare with previous version

added 2 commits <ul><li>e7b8690c - Simplify some queries</li><li>abbd2ccd - Add prefetch option in week view</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3297&start_sha=66071535d0edc369137f0355c469afbc895ba86b)
hansegucker commented 2020-09-11 11:22:12 +02:00
Author
Owner
Copy link

added 6 commits

  • abbd2ccd...5833dfa2 - 3 commits from branch master
  • 120193ca - Merge branch 'master' into 73-add-rules-and-permissions
  • cff25c97 - Add permissions for "My groups"
  • 50629966 - Merge branch '73-add-rules-and-permissions' into feature/performance-optimizations

Compare with previous version

added 6 commits <ul><li>abbd2ccd...5833dfa2 - 3 commits from branch <code>master</code></li><li>120193ca - Merge branch &#39;master&#39; into 73-add-rules-and-permissions</li><li>cff25c97 - Add permissions for &quot;My groups&quot;</li><li>50629966 - Merge branch &#39;73-add-rules-and-permissions&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3300&start_sha=abbd2ccd8441c89569f0b91d4b2b2f638d37e781)
hansegucker commented 2020-09-11 11:24:22 +02:00
Author
Owner
Copy link

added 2 commits

  • 34ef43d1 - Add menu validator for "My groups"
  • 6e333849 - Merge branch '73-add-rules-and-permissions' into feature/performance-optimizations

Compare with previous version

added 2 commits <ul><li>34ef43d1 - Add menu validator for &quot;My groups&quot;</li><li>6e333849 - Merge branch &#39;73-add-rules-and-permissions&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3302&start_sha=50629966c2e007dbc6b8651caca17c06d25e6cf7)
hansegucker commented 2020-09-27 15:24:10 +02:00
Author
Owner
Copy link

added 6 commits

  • 6e333849...dad563e6 - 4 commits from branch master
  • ed1fede6 - Merge branch 'master' into 73-add-rules-and-permissions
  • 82107ae6 - Merge branch '73-add-rules-and-permissions' into feature/performance-optimizations

Compare with previous version

added 6 commits <ul><li>6e333849...dad563e6 - 4 commits from branch <code>master</code></li><li>ed1fede6 - Merge branch &#39;master&#39; into 73-add-rules-and-permissions</li><li>82107ae6 - Merge branch &#39;73-add-rules-and-permissions&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3403&start_sha=6e333849266560403f0852e8ebfe3a2c134efe52)
hansegucker commented 2020-10-15 17:52:47 +02:00
Author
Owner
Copy link

added 3 commits

  • ee72ebc2 - Fix problems with update_or_create and prefetching
  • 091caa94 - Merge branch 'fix/update-or-create' into 73-add-rules-and-permissions
  • 9d51bbfb - Merge branch '73-add-rules-and-permissions' into feature/performance-optimizations

Compare with previous version

added 3 commits <ul><li>ee72ebc2 - Fix problems with update_or_create and prefetching</li><li>091caa94 - Merge branch &#39;fix/update-or-create&#39; into 73-add-rules-and-permissions</li><li>9d51bbfb - Merge branch &#39;73-add-rules-and-permissions&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3501&start_sha=82107ae6322c4b2d6be458b6145e11fb3bea8db4)
hansegucker commented 2020-10-15 17:58:22 +02:00
Author
Owner
Copy link

added 1 commit

  • 56806b7f - Use classproperty from functional

Compare with previous version

added 1 commit <ul><li>56806b7f - Use classproperty from functional</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3502&start_sha=9d51bbfb0344740e36a76224cafa6c3916c51a67)
hansegucker commented 2020-10-25 12:22:46 +01:00
Author
Owner
Copy link

added 12 commits

  • 56806b7f...50db5254 - 11 commits from branch master
  • c32c3d04 - Merge branch 'master' into feature/performance-optimizations

Compare with previous version

added 12 commits <ul><li>56806b7f...50db5254 - 11 commits from branch <code>master</code></li><li>c32c3d04 - Merge branch &#39;master&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3529&start_sha=56806b7fb1983f286ef9c321c041e6d1a1d11b9d)
hansegucker commented 2020-10-25 12:23:02 +01:00
Author
Owner
Copy link

changed the description

changed the description
hansegucker commented 2020-10-25 12:23:45 +01:00
Author
Owner
Copy link

Please review, @nik!

Please review, @nik!
hansegucker commented 2020-10-25 12:23:45 +01:00
Author
Owner
Copy link

unmarked as a Work In Progress

unmarked as a **Work In Progress**
nik commented 2020-10-25 17:19:52 +01:00
Owner
Copy link

The last comma here reads like a syntax error.

The last comma here reads like a syntax error.
nik commented 2020-10-25 17:19:54 +01:00
Owner
Copy link
Help on built-in function all in module builtins:

all(iterable, /)
    Return True if bool(x) is True for all values x in the iterable.
    
    If the iterable is empty, return True.

Also, what is this good for?

``` Help on built-in function all in module builtins: all(iterable, /) Return True if bool(x) is True for all values x in the iterable. If the iterable is empty, return True. ``` Also, what is this good for?
nik commented 2020-10-25 17:19:55 +01:00
Owner
Copy link

Are you sure using a loop will improve performance?

Are you sure using a loop will improve performance?
hansegucker commented 2020-10-25 17:29:46 +01:00
Author
Owner
Copy link

But it isn't a syntax error and it's formatted in this way by black.

But it isn't a syntax error and it's formatted in this way by black.
hansegucker commented 2020-10-25 17:36:36 +01:00
Author
Owner
Copy link

Ok, I will rename it. It should centralize access on these common-used models with a low count of entries, so that the results can be cached.

Ok, I will rename it. It should centralize access on these common-used models with a low count of entries, so that the results can be cached.
hansegucker commented 2020-10-25 17:36:37 +01:00
Author
Owner
Copy link

It definitely reduces database queries, because the values used by the loop are prefetched. The reduced count of database queries beats the time a Python loop needs.

It definitely reduces database queries, because the values used by the loop are prefetched. The reduced count of database queries beats the time a Python loop needs.
nik commented 2020-10-25 18:07:28 +01:00
Owner
Copy link

I do not see how this reduces database queries.

I do not see how this reduces database queries.
nik commented 2020-10-25 18:12:47 +01:00
Owner
Copy link

Have you researched what others do to cache querysets?

Have you researched what others do to cache querysets?
nik commented 2020-10-25 18:15:11 +01:00
Owner
Copy link

We might want https://github.com/noripyt/django-cachalot

We might want https://github.com/noripyt/django-cachalot
hansegucker commented 2020-10-25 18:35:05 +01:00
Author
Owner
Copy link

Filter doesn't make use of prefetched values in related querysets, all does.

`Filter` doesn't make use of prefetched values in related querysets, `all` does.
hansegucker commented 2020-10-25 18:35:06 +01:00
Author
Owner
Copy link

Yes, but it's always a decision between a) we cache everything or b) we cache nothing. So that change here should be seen only as a workaround.

Yes, but it's always a decision between a) we cache everything or b) we cache nothing. So that change here should be seen only as a workaround.
hansegucker commented 2020-10-25 18:35:21 +01:00
Author
Owner
Copy link

Yes, but then in core.

Yes, but then in core.
hansegucker commented 2020-10-25 18:35:44 +01:00
Author
Owner
Copy link

created #116 to continue this discussion

created #116 to continue this discussion
hansegucker commented 2020-10-25 18:35:44 +01:00
Author
Owner
Copy link

mentioned in issue #116

mentioned in issue #116
hansegucker commented 2020-10-26 19:34:34 +01:00
Author
Owner
Copy link

added 29 commits

  • c32c3d04...8e855851 - 28 commits from branch master
  • 4e460d55 - Merge branch 'master' into feature/performance-optimizations

Compare with previous version

added 29 commits <ul><li>c32c3d04...8e855851 - 28 commits from branch <code>master</code></li><li>4e460d55 - Merge branch &#39;master&#39; into feature/performance-optimizations</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3532&start_sha=c32c3d042d4e9d1050fd22883dc7f02326649c8d)
nik commented 2020-10-27 16:29:46 +01:00
Owner
Copy link

Decided to use cachalot and "cache everything", so please remove all these workaround propertes from this MR.

Decided to use cachalot and "cache everything", so please remove all these workaround propertes from this MR.
nik commented 2020-10-27 16:50:28 +01:00
Owner
Copy link

Please remove in favour of cachalot.

Please remove in favour of cachalot.
nik commented 2020-10-27 16:50:29 +01:00
Owner
Copy link

Please use a filter() for readability (and to account for any improvements on the RelatedManager filter API in spe).

Please use a `filter()` for readability (and to account for any improvements on the `RelatedManager` filter API in spe).
nik commented 2020-10-27 16:50:29 +01:00
Owner
Copy link

Please remove in favour of cachalot.

Please remove in favour of cachalot.
nik commented 2020-10-27 16:50:30 +01:00
Owner
Copy link

Please remove in favour of cachalot.

Please remove in favour of cachalot.
nik commented 2020-10-27 16:50:31 +01:00
Owner
Copy link

Please us a filter(), as discussed above.

Please us a `filter()`, as discussed above.
hansegucker commented 2020-10-27 17:05:02 +01:00
Author
Owner
Copy link

resolved all threads

resolved all threads
hansegucker commented 2020-10-27 17:05:10 +01:00
Author
Owner
Copy link

resolved all threads

resolved all threads
hansegucker commented 2020-10-27 17:06:22 +01:00
Author
Owner
Copy link

changed this line in version 23 of the diff

changed this line in [version 23 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3535&start_sha=4e460d55b9fa6d1fab9eee8fc0594209e8904891#780f294f550e4cd96aa99e7069b309c8b842f1d2_41_39)
hansegucker commented 2020-10-27 17:06:24 +01:00
Author
Owner
Copy link

changed this line in version 23 of the diff

changed this line in [version 23 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3535&start_sha=4e460d55b9fa6d1fab9eee8fc0594209e8904891#bd8664958a1ee12dd2af439715dcc93cbd19732d_53_49)
hansegucker commented 2020-10-27 17:06:24 +01:00
Author
Owner
Copy link

changed this line in version 23 of the diff

changed this line in [version 23 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3535&start_sha=4e460d55b9fa6d1fab9eee8fc0594209e8904891#70ecb348d7a796878a62bf6188416dcc55b207e6_182_180)
hansegucker commented 2020-10-27 17:06:24 +01:00
Author
Owner
Copy link

changed this line in version 23 of the diff

changed this line in [version 23 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3535&start_sha=4e460d55b9fa6d1fab9eee8fc0594209e8904891#70ecb348d7a796878a62bf6188416dcc55b207e6_248_252)
hansegucker commented 2020-10-27 17:06:25 +01:00
Author
Owner
Copy link

changed this line in version 23 of the diff

changed this line in [version 23 of the diff](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3535&start_sha=4e460d55b9fa6d1fab9eee8fc0594209e8904891#780f294f550e4cd96aa99e7069b309c8b842f1d2_222_212)
hansegucker commented 2020-10-27 17:06:26 +01:00
Author
Owner
Copy link

added 3 commits

Compare with previous version

added 3 commits <ul><li>8e6c9320 - Remove legacy caching methods</li><li>29b5aa3a - Use filter in favour of for loops</li><li>7d767261 - Reformat</li></ul> [Compare with previous version](/AlekSIS/official/AlekSIS-App-Alsijil/-/merge_requests/90/diffs?diff_id=3535&start_sha=4e460d55b9fa6d1fab9eee8fc0594209e8904891)
nik commented 2020-10-27 17:09:27 +01:00
Owner
Copy link

mentioned in commit 1ed6789523

mentioned in commit 1ed6789523053e6d2089ee4a2b7bddb67b014c71
nik merged commit 1ed6789523 into master 2020-10-27 17:09:28 +01:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
Security

   
TeX
Needs LaTeX template

  
auto-update
  
board
done

  
board
ready

  
board
todo

  
check
delete-eslint-rc-js

  
check
update-builddeps-package-json

  
check
update-eslint-rc-js

  
check
update-gitignore

  
check
update-merge-request-template

  
check
update-prettier-ignore

  
check
update-pyproject-toml

  
check
update-renovate-json

  
check
update-tox-ini

  
part
backend
 Backend (view code, models, logic, etc.)

  
part
ci

  
part
docs
 Issue or addition to the documentation

  
part
frontend
 Frontend (templates, etc.)

  
part
i18n
 Translation or internationalisation issue

  
part
non-technical
 Non-technical (policy, maintenance, etc.) issue or discussion

  
part
packaging
 Issue concerning the packaging of AlekSIS and components

  
prio
1
 Priority High

  
prio
2
 Priority Medium

  
prio
3
 Priority Low

  
release-mr-5.x
  
size
large

  
size
medium

  
size
small

  
source
customer
 Requested by a customer or affiliated user

  
source
customer::fsmw

  
source
customer::fss

  
source
customer::teckids

  
source
downstream
 Issue or merge request from other projects using AlekSIS or parts of it

  
type
breaking

  
type
bug
 Bug

  
type
feature
 New feature

  
type
refactoring

  
workflow
blocked
 Issue is blocked by another issue

  
workflow
confirmed
 The bug is reproducible and will be fixed

  
workflow
current-todo

  
workflow
discussing
 Needs to be discussed before implementation

  
workflow
new-app
 Requires new AlekSIS app, which has to be created

  
workflow
wontfix
 The bug is not reproducible or the feature was decided against

  
source::aleksolotl
No labels
Security
TeX
auto-update
board
done
board
ready
board
todo
check
delete-eslint-rc-js
check
update-builddeps-package-json
check
update-eslint-rc-js
check
update-gitignore
check
update-merge-request-template
check
update-prettier-ignore
check
update-pyproject-toml
check
update-renovate-json
check
update-tox-ini
part
backend
part
ci
part
docs
part
frontend
part
i18n
part
non-technical
part
packaging
prio
1
prio
2
prio
3
release-mr-5.x
size
large
size
medium
size
small
source
customer
source
customer::fsmw
source
customer::fss
source
customer::teckids
source
downstream
type
breaking
type
bug
type
feature
type
refactoring
workflow
blocked
workflow
confirmed
workflow
current-todo
workflow
discussing
workflow
new-app
workflow
wontfix
source::aleksolotl
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
aleksis/AlekSIS-App-Alsijil!480
No description provided.