What is the recommended way to handle paginated results to avoid duplicates?

Support
1

As per Weaviate docs:

The pagination setup is not stateful. If the database state has changed between retrieving two pages there is no guarantee that your pages cover all results.

I am having an issue right now where, when someone publishes their new Player X then another Player Y ranking could go either higher or lower from the search relevance.

If Player Y (ranked 89 before the update) has already appeared in page 81-90 (currently loaded) and after an update this player went to ranked 92 thus, this player will be part of page 91-100 which will appear like a duplicate record for the user viewing our search page.

It is more noticeable on our side since we are using infinite scroll (business rule).

Is there like a recommended way on how to handle this on the application side? Any suggestions to move forward?

2

Hi @junbetterway - I’ve moved this to Support as I think that fits better with the topic.

As to your question, I’m not sure what the best solution will be - I will ask around and get back to you.

1 Like
3

## Create an object with ID

creating objects by ID, and then in your backend you can note which objects were shown and which were not yet based on there unique ID

then you only return object not shown yet,

so even when reranking happens, you do not show duplicates

4

Thanks this is what I ended up doing on the client-side (html/js), since our backend only has the response copy of the current page result. So far no indications on slow performance when tracking unique IDs. Cheers!