[Autocomplete] support pagination on Asynchronous requests

[vadimka123]

Would be very nice to add any ability for support pagination on Asynchronous requests
Many resources have limitations on requests and we can not get all results at once
Also reading everything page by page can lead to exceed limit of requests and it will be too slow

We would likely need to extend the API:

diff --git a/packages/material-ui/src/Autocomplete/Autocomplete.d.ts b/packages/material-ui/src/Autocomplete/Autocomplete.d.ts
index a2929c4867..39fa7130b3 100644
--- a/packages/material-ui/src/Autocomplete/Autocomplete.d.ts
+++ b/packages/material-ui/src/Autocomplete/Autocomplete.d.ts
@@ -143,6 +143,11 @@ export interface AutocompleteProps<
    * @default false
    */
   loading?: boolean;
+  /**
+   * If `true`, the `loadingMoreText` message is displayed at the bottom of the displayed options.
+   * @default false
+   */
+   loadingMore?: boolean;
   /**
    * Text to display when in a loading state.
    *
@@ -150,6 +155,13 @@ export interface AutocompleteProps<
    * @default 'Loading…'
    */
   loadingText?: React.ReactNode;
+  /**
+   * Text to display when in a loading more state.
+   *
+   * For localization purposes, you can use the provided [translations](/guides/localization/).
+   * @default 'Loading more…'
+   */
+  loadingMoreText?: React.ReactNode;
   /**
    * The maximum number of tags that will be visible when not focused.
    * Set `-1` to disable the limit.

Benchmark

• feat(pagination): add async pagination JedWatson/react-select#1237
• https://select2.org/data-sources/ajax#pagination

[oliviertassinari]

@vadimka123 I'm not sure to understand the use case. From my perspective, pagination should not be necessary. Either you request the data incrementally and add them to the list of options, then once the list is fully loaded, you enable the search. Or you really have too much data and you opt for an instant search like call API, for each keystroke. What do you think?

[vadimka123]

@oliviertassinari
It's can be million of options for AutoComplete

you can see example of this feature in select2 library
https://select2.org/data-sources/ajax#pagination

react-select has onMenuScrollToBottom callback and it's help to build bussiness logic for pagination in async requests

Even ordinary callback would really help

[oliviertassinari]

@vadimka123 Thanks for the link. Ok so if you have millions of options, you would need to perform a query per keystroke, similar to https://material-ui.com/components/autocomplete/#google-maps-place. What's your API like?

[vadimka123]

@oliviertassinari, various CRMs like Salesforce, Pardot, Eloqua, Marketo

Also I updated prev comment
Please, note

[oliviertassinari]

@vadimka123 Do you have an example of such API so we can have a look at how we can best handle it?

[vadimka123]

@oliviertassinari
Pardot, Eloqua and Marketo - have only paid instances
In Salesforce you can register free dev instance, you can read docs (https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_query.htm), but need a lot of work for API realization

[oliviertassinari]

@vadimka123 Ok, I see your point. You have an autocomplete like API that returns x results and you would like to allow users to scroll down the list of options instead of asking them to refine the query with something more specific that returns fewer results, below the pagination logic.

If it's accurate, I would challenge the use case for it in the first place. Forcing users to refind the query might yield a better UX. It might not be very important to support. However, we can explore it further. They are multiple elements to uncover here:

• The component exposes enough API surface to implement this but it would require extra work each time, for each developer.
• We would need to detect when we are about to reach the end of the scroll.
• We would probably want to add this in the public API, it's definitely required to explore paginated data, e.g. onScrollToBottom.
• We would need to display a loader when we reach the bottom of the listbox to indicate that more items are being loaded.
• The final step is the network request. There is a wide variety of possible endpoints people can query, from REST to GraphQL APIs.

[oliviertassinari]

So, if you provide a custom ListboxComponent component or you can detect the scroll-bottom with:

<Autocomplete
  ListboxProps={{
    onScroll: (event: React.SyntheticEvent) => {
      const listboxNode = event.currentTarget;
      if (listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight) {
        loadMoreResults()
      }
    }
  }}
/>

[vadimka123]

@oliviertassinari
I think you should not override Material UI by business logic for this case
Business logic (like API requests) should make developer who uses your library
You should provide ability for make this business logic, also add very basic example would be great

Callback when menu is scrolled. At this point you offer one option to implement this. It's great
Also user can move to end of list of options by using key buttons. It's also should be handled
It's require more work for developer, but looks like it's possible now

---

Also there is one big catch
Already have a bad experience with this in react-select
Part of users can enter "End" key button and this automatically move user to last option (maybe exists other features for make this), ability to disable\prevent this feature would really help

---

Per documentation you have loading state. It's can be used for add loading indicator as last option if exists other options?

[oliviertassinari]

@vadimka123 Yeah, I'm not opposed to a onScrollToBottom prop, but I would like to get more feedback that it's frequently needed. I suspect it's not.

---

Part of users can enter "End" key button and this automatically move user to last option (maybe exists other features for make this), ability to disable\prevent this feature would really help

WAI-ARIA makes the Home and End key handling optional: https://www.w3.org/TR/wai-aria-practices/#combobox. Could you share more on what it can be problematic? I think that we can consider such an option.

---

Per documentation you have loading state. It's can be used for add loading indicator as last option if exists other options?

I don't know. I guess yes?

[vadimka123]

@oliviertassinari
If I use custom ListboxComponent I can't pass any props to this component

[jedwards1211]

@oliviertassinari I'm not sure it's a great idea to only provide onScrollToBottom and just load more and more data. I think it would be better to include onScrollToTop if pagination is implemented. I guess it's not likely with an autocomplete that someone would keep scrolling until the browser runs out of memory, but I generally want to guarantee that any paginated view has bounded memory usage. A lot of infinite scroll examples I've seen, even in react-virtualized, aren't responsible about this.

[oliviertassinari]

I'm still confused about the problem we are trying to solve here. So far, I have seen the following mentions:

1. "A have millions of options".
   In such a case, you will need to run a query to the backend for every throttled keystroke. We can't expect the end-users to scroll on many options. After 10, I would start to question the human attention and focus capability. For instance, Google displays at most 10 searches autocomplete suggestions.
2. "My backend displays x records, use pagination for more".
   In such a case, I would recommend to first try to increase the number of records returned per page. I would expect 100 returned records on page 1 to be way more than enough.
   If it's not possible (I would be surprised), developers could eagerly load multiple pages.

In both cases, I believe the Autocomplete components don't need any changes. Is it accurate? What did I miss?

---

To summarize, we are waiting for a compelling reason. Why would an end-user or a UX designer would want to implement this behavior?

[luisanton-io]

<Autocomplete
  ListboxProps={{
    onScroll: (event: React.SyntheticEvent) => {
      const listboxNode = event.currentTarget;
      if (listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight) {
        loadMoreResults()
      }
    }
  }}
/>

Hello Olivier, this has just been useful for me as well.
In my use case, Users are Artists and are supposed to look for themselves in music platforms databases with their stage name.
What happens is that many choose short names and, unfortunately, more popular artists are showing up first despite them having longer names: the only solution is letting them scroll as much as needed.

Your solution was easy enough, however, I also think that adding this as a specific feature is not pointless at all.
Thank you!
L

[oliviertassinari]

I'm adding the "good to take" label as it seems easy to implement. We could also consider a scrollEndThreshold prop.

@DanailH has recently added a similar feature in the data grid component under mui/mui-x#1199.

[vtassios]

I also have a big amount of data that I need to fetch. So it would be good to have a prop that will load more data onScroll, to prevent delays.

[picosam]

I also second the above comment.

[anushaNemilidinne]

Hello! Is anyone working on this? This is really a useful feature. I achieved the pagination using react-window-infinite-loader with Autocomplete.

[thefenry]

I agree this would be a useful. I had implemented the solution offered previously but after migrating to v5, the items scroll back to top on loading more items.

[roman-kuleshov]

I agree this would be a useful. I had implemented the solution offered previously but after migrating to v5, the items scroll back to top on loading more items.

Hello! Have you maybe found a solution to keep the position of the scroll after loading more items?

[francisco-polaco]

Hello everyone!
Anyone working on this in 2022? This is an extremely useful feature.

[rrfaria]

I have an use case similar and autocomplete functionality is handled all on backend and listing is paginated

so when user click it loads this request

users?page=0&query=

and when user scroll to end on list
it trigers new resquest if it is not last page

users?page=1&query=

if user type anything on input it trigers another resquest

users?page=0&query=JohnDoe

users are grouped by rule to it I can use groupBy prop
but to do query and infinite scrolling I couldn't find anything

I could use this workaround here
but when it loads more items and add to state, component will close options because state changes

Is there any idea how can I do it using this component?

[ZeeshanTamboli]

We are working on adding a onScrollToBottom prop in #35653 for Material UI and Joy UI. You can try it out with the preview package by replacing the package versions in your package.json with the preview links:

Material UI:

"@mui/material": "https://pkg.csb.dev/mui/material-ui/commit/f4315147/@mui/material"

Joy UI:

"@mui/joy": https://pkg.csb.dev/mui/material-ui/commit/f4315147/@mui/joy

Let us know for any feedback!

[crow7m]

Good day, is there any update on this one?
thank you

[hamzashakir99]

Good day, is there any update on this one? thank you

I think there is no update, i worked one day but did not find solution with mui tag

[vadimka123]

<Autocomplete
  ListboxProps={{
    onScroll: (event: React.SyntheticEvent) => {
      const listboxNode = event.currentTarget;
      if (listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight) {
        loadMoreResults()
      }
    }
  }}
/>

This solution working for us
Maybe need documentation with this case, but all working good

Maybe close ?

[roman-kuleshov]

<Autocomplete
  ListboxProps={{
    onScroll: (event: React.SyntheticEvent) => {
      const listboxNode = event.currentTarget;
      if (listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight) {
        loadMoreResults()
      }
    }
  }}
/>

This solution working for us Maybe need documentation with this case, but all working good

Maybe close ?

I'm using the same solution and it seems that it doesn't scroll the list to the very top after loading a new batch of results anymore 👍

    "@mui/material": "5.14.3",

[Duskfen]

<Autocomplete
  ListboxProps={{
    onScroll: (event: React.SyntheticEvent) => {
      const listboxNode = event.currentTarget;
      if (listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight) {
        loadMoreResults()
      }
    }
  }}
/>

This solution working for us Maybe need documentation with this case, but all working good
Maybe close ?

I'm using the same solution and it seems that it doesn't scroll the list to the very top after loading a new batch of results anymore 👍

    "@mui/material": "5.14.3",

i used following fix to work around the scrolling issue:
in the onScroll function i store the listboxNode and its scrolltop in a ref,
then i call the fetching function which updates my data
(

          onScroll: (event: React.SyntheticEvent) => {
            const listboxNode = event.currentTarget;
            if (
              listboxNode.scrollTop + listboxNode.clientHeight ===
              listboxNode.scrollHeight
            ) {
              persistedListBox.current = listboxNode;
              persistedScrollTop.current = listboxNode.scrollTop;
              debouncedFetchTestObjects(inputValue);
            }
          },
        }}

)

in a useEffect I listen to this data and set the scroll to the old value:

  useEffect(() => {
    if (persistedListBox.current) {
      persistedListBox.current.scrollTo({
        top: persistedScrollTop.current,
      });
      persistedListBox.current = null;
    }
  }, [fetchedTestObjectTypes]);

if you can await your loadMoreResults() function i would recommend you not solving this over useEffect and useRefs, but await the loadMoreResults() and then scrollTo afterwards. (Note: make sure to store the scrollTop in a variable before calling your fetching function as it will get overwritte otherwise)

[Nic-S]

Hello, any news?

[thangnguyen22y]

So, if you provide a custom ListboxComponent component or you can detect the scroll-bottom with:

<Autocomplete
  ListboxProps={{
    onScroll: (event: React.SyntheticEvent) => {
      const listboxNode = event.currentTarget;
      if (listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight) {
        loadMoreResults()
      }
    }
  }}
/>

This solution works for me! ("@mui/material": "^5.15.1")
Only one issue: if my browser is zoomed out, this formula is not correct anymore:
listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight.

[arunmmanoharan]

Working sample.

https://stackblitz.com/edit/react-fb9r8f?file=Demo.tsx

[jannnik]

Only one issue: if my browser is zoomed out, this formula is not correct anymore:
listboxNode.scrollTop + listboxNode.clientHeight === listboxNode.scrollHeight.

A solution for this can be to use Math.round, but it also does not work for very big zoom levels.

if (Math.round(listboxNode.scrollTop + listboxNode.clientHeight) === listboxNode.scrollHeight) {

[afilp]

@arunmmanoharan thank you, but when I search for an item that is not on the initial loaded list, then this does not appear. How can we have both the "list" logic but also the "search" logic to work for the all the values of the backend table?