You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What we're looking for specifically here are client side limitations on document size, and memory footprint.
Something that is commonly cited is a single response document limitation is about 10 MB. Without any assumptions around client side streaming support, that would imply working over results in the 10,000s range when dealing with more than just the ids. Also touched on here #41036 (reply in thread)
Dealing with just the ids should imply the ability for the client handle on the order of 250000 results.
These limitations influence whether, and how, we choose to follow-up with projection support only, or design 1 (the current state of #48289), 2, 3 from #48286 (comment) - or even selectively employing a strategy based upon the resource type.
To continue with 1 means making the UI design geared towards providing users with the ability to select more representative subsets to paginate over, rather than providing arbitrary pagination controls across the entire data set. This also probably means providing at least default limits on our searchable list endpoints to ensure that clients fetch only reasonable subsets.
Note that if we continue with 1 such that server side ordering is implemented, ordered querying over the sort columns is possible (not covered by the initial querying work), and a limit / max results option is available (presumably this is part also already part of the scope for number 1), then clients may optional choose to implement the strategy behind 2 - that is use a unique sort, limit, and predicates to control which subset they are fetching. The biggest limitation here is that no skipping is possible, you have to scroll to determine what values represent the pages. If the querying api is limited to only equality, then explicit pagination controls are needed for number 2 instead.
What we're looking for specifically here are client side limitations on document size, and memory footprint.
I'd say not only the client side limitations. We want to see server side implications too – if the 250000 resources is even feasible to be returned in a single response, even if it's just IDs.
This also probably means providing at least default limits on our searchable list endpoints to ensure that clients fetch only reasonable subsets.
You mean to enforce maximu size of the returned result set? +1 to that. We could also indicate e.g. by the response code that not all results are returned.
If the querying api is limited to only equality, then explicit pagination controls are needed for number 2 instead.
I'd say not only the client side limitations. We want to see server side implications too – if the 250000 resources is even feasible to be returned in a single response, even if it's just IDs.
The most evident server side issue is with inifinispan cache managed entities. It is acting like a session level persistence context, so when you read too many entities, it grows too large. While there are not currently managed clients, there are managed roles. Mitigation strategies:
we already have some controls around batch mode (for import) to prevent creating managed entries. That could be extended for read-only streaming scenarios - run the query in batch mode, and make sure we suppress creating even more managed entities.
add the ability, similar to some existing PersistenceContexts logic, to flush managed caches per client - the expectation being that we don't reuse any of entries across clients, is that correct?
add hard limits to how many results can be fetched - but I don't want us to do this until we better unstand what the values should be. Also I don't want to allow for clients that can use streaming to do so as much as possible.
I'll double check, but I don't believe there are other places where one of our layers is doing something odd - like an in-memory sort.
With that in mind I created an endpoint to perform a clean projection - just the id and protocol (more on that below [1]) - and also utilized the streaming client logic (#47611). I could send repeatedly 200000000 dummy clients in one request without appreciable memory utilization differences on the client or server compared to 200000, or 2000 in a request.
Having tighter transaction timeouts and sensible defaults and/or maximums on the number of results returned will help keep queries like these from starving out other work
Client-side handling - I’ve seen 10 MB as a practical limit in a single document, we probably need some empirical confirmation. @edewit do you have any idea / guidelines here? See my previous comments for some rough guesstimates.
[1] Much of the model logic and underlying entity logic is lazy wrt collections, so it is typically enough to just not read those fields from the Model to prevent the join fetching. The simplest projection implementation we're looking to do is in the model mapping layer to omit the fields we don't want. However some fields on the representation can be computed / fixed, like protocol, and so could still show up in the output document. Omitting the protocol from the output in particular also has the issue that we can't deserialize with Java client. It either needs to always be included, or we need to introduce a non-abstract base type that can be used in situations where the protocol is omitted. I'll also add this comment to the projection issue.
You mean to enforce maximu size of the returned result set? +1 to that. We could also indicate e.g. by the response code that not all results are returned.
Yes, but I'd like to fully understand why we're picking our limits before proposing something. Changing the response code or even using a Stream subclass to convey more information - such as "hasMore" and if an error occurred (#34857)
Not sure I follow. Can you elaborate?
A unique sort order cursor relies upon other operators like greater / less than as it scrolls through the pages. After the first page you are asking for something like - "fetch the next 100 clients ordered by name with name > foo". If we directly support greater than (forward scolling) or lesser than (reverse scolling) in the query api this becomes expressable directly by the client, instead of requiring specific pagination control parameters.
So there isn't a single "max response size" for fetch in a browser. Parsing a huge JSON array can freeze the tab for seconds and users experience that as "the UI hung" long before anything "officially" fails. Memory doubles with parse, so it's risky. How about we use streaming and use NDJSON ( so that each chuck is valid JSON ) so that the client could stop / abort the loading of IDs after it received the page it wants and then it would still be parse-able.
To recap where things are - I wanted to use this issue to walk through the considerations around each style of pagination that clients may be able do. That starts off with two flavors of fully client side (where the pagination is then a batching or presentational concern client side):
I think we've roughly answered here that even relatively large results are still usable - in particular if the client supports a streaming parser (see #47611). What I would like as the take-away here is that we can support either a larger default or cap than than just 100 results. Some scim systems seem to allow for a configurable max limit value.
Given that we have a default ordering if in general a default limit is enforced, the results need to indicate when they are partial - and the scenario will degrade to one of the other pagination styles.
The next pagination style, which we haven't talked about here yet, is cursoring. Here a unique sort is specified, then pages are determined by using > or < queries along with limit. Here again this can be done with quering support already exposed by scim. This is suitable for simple navigation, not for adhoc skipping of pages. Usage of an offset is not required, nor is returning a total count. This style of pagination, expecially if the count is omitted, should offer better performance than using an offset / limit. We can keep this issue open to validate that, or create a new issue if that seems better.
Finally offset / limit (with the default or explicit sort) can be used to get the desired page.
Another cross-cutting performance concern, mentioned in #41036, is around federated resources - but that could be handled by a different issue.
Keeping this open as a generic reminder to review various pagination approaches in terms of performance. This includes pagination over federated resources, pointer based pagination, client side pagination etc. This might result into just a docs issue.
Activity
vmuzikar commentedon Apr 21, 2026
This issue will validate if the approach described in #48289 is feasible.
shawkins commentedon Apr 21, 2026
What we're looking for specifically here are client side limitations on document size, and memory footprint.
Something that is commonly cited is a single response document limitation is about 10 MB. Without any assumptions around client side streaming support, that would imply working over results in the 10,000s range when dealing with more than just the ids. Also touched on here #41036 (reply in thread)
Dealing with just the ids should imply the ability for the client handle on the order of 250000 results.
These limitations influence whether, and how, we choose to follow-up with projection support only, or design 1 (the current state of #48289), 2, 3 from #48286 (comment) - or even selectively employing a strategy based upon the resource type.
To continue with 1 means making the UI design geared towards providing users with the ability to select more representative subsets to paginate over, rather than providing arbitrary pagination controls across the entire data set. This also probably means providing at least default limits on our searchable list endpoints to ensure that clients fetch only reasonable subsets.
Note that if we continue with 1 such that server side ordering is implemented, ordered querying over the sort columns is possible (not covered by the initial querying work), and a limit / max results option is available (presumably this is part also already part of the scope for number 1), then clients may optional choose to implement the strategy behind 2 - that is use a unique sort, limit, and predicates to control which subset they are fetching. The biggest limitation here is that no skipping is possible, you have to scroll to determine what values represent the pages. If the querying api is limited to only equality, then explicit pagination controls are needed for number 2 instead.
cc @vmuzikar
vmuzikar commentedon Apr 22, 2026
I'd say not only the client side limitations. We want to see server side implications too – if the 250000 resources is even feasible to be returned in a single response, even if it's just IDs.
You mean to enforce maximu size of the returned result set? +1 to that. We could also indicate e.g. by the response code that not all results are returned.
Not sure I follow. Can you elaborate?
shawkins commentedon Apr 22, 2026
The most evident server side issue is with inifinispan cache managed entities. It is acting like a session level persistence context, so when you read too many entities, it grows too large. While there are not currently managed clients, there are managed roles. Mitigation strategies:
I'll double check, but I don't believe there are other places where one of our layers is doing something odd - like an in-memory sort.
With that in mind I created an endpoint to perform a clean projection - just the id and protocol (more on that below [1]) - and also utilized the streaming client logic (#47611). I could send repeatedly 200000000 dummy clients in one request without appreciable memory utilization differences on the client or server compared to 200000, or 2000 in a request.
For me the limitations wrt to pagination are:
[1] Much of the model logic and underlying entity logic is lazy wrt collections, so it is typically enough to just not read those fields from the Model to prevent the join fetching. The simplest projection implementation we're looking to do is in the model mapping layer to omit the fields we don't want. However some fields on the representation can be computed / fixed, like protocol, and so could still show up in the output document. Omitting the protocol from the output in particular also has the issue that we can't deserialize with Java client. It either needs to always be included, or we need to introduce a non-abstract base type that can be used in situations where the protocol is omitted. I'll also add this comment to the projection issue.
Yes, but I'd like to fully understand why we're picking our limits before proposing something. Changing the response code or even using a Stream subclass to convey more information - such as "hasMore" and if an error occurred (#34857)
A unique sort order cursor relies upon other operators like greater / less than as it scrolls through the pages. After the first page you are asking for something like - "fetch the next 100 clients ordered by name with name > foo". If we directly support greater than (forward scolling) or lesser than (reverse scolling) in the query api this becomes expressable directly by the client, instead of requiring specific pagination control parameters.
edewit commentedon Apr 23, 2026
So there isn't a single "max response size" for fetch in a browser. Parsing a huge JSON array can freeze the tab for seconds and users experience that as "the UI hung" long before anything "officially" fails. Memory doubles with parse, so it's risky. How about we use streaming and use NDJSON ( so that each chuck is valid JSON ) so that the client could stop / abort the loading of IDs after it received the page it wants and then it would still be parse-able.
14 remaining items
shawkins commentedon May 15, 2026
#49028 demonstrates how to make our streaming end-points more memory safe, basically addressing what was mentioned in #48328 (comment)
shawkins commentedon May 21, 2026
To recap where things are - I wanted to use this issue to walk through the considerations around each style of pagination that clients may be able do. That starts off with two flavors of fully client side (where the pagination is then a batching or presentational concern client side):
or, scim query syntax can be used for this as is.I think we've roughly answered here that even relatively large results are still usable - in particular if the client supports a streaming parser (see #47611). What I would like as the take-away here is that we can support either a larger default or cap than than just 100 results. Some scim systems seem to allow for a configurable max limit value.
Given that we have a default ordering if in general a default limit is enforced, the results need to indicate when they are partial - and the scenario will degrade to one of the other pagination styles.
The next pagination style, which we haven't talked about here yet, is cursoring. Here a unique sort is specified, then pages are determined by using > or < queries along with limit. Here again this can be done with quering support already exposed by scim. This is suitable for simple navigation, not for adhoc skipping of pages. Usage of an offset is not required, nor is returning a total count. This style of pagination, expecially if the count is omitted, should offer better performance than using an offset / limit. We can keep this issue open to validate that, or create a new issue if that seems better.
Finally offset / limit (with the default or explicit sort) can be used to get the desired page.
Another cross-cutting performance concern, mentioned in #41036, is around federated resources - but that could be handled by a different issue.
cc @edewit @vmuzikar
vmuzikar commentedon Jun 8, 2026
Keeping this open as a generic reminder to review various pagination approaches in terms of performance. This includes pagination over federated resources, pointer based pagination, client side pagination etc. This might result into just a docs issue.