The easier thing to do would be to first check if you can use their api first if it is exposed in some way even if you have to login with authentication and use the api to page.
If you for some reason you cannot because the data is rendered from multiple apis you would need to potentially call more JavaScript calls on the page. Basically simulating scrolling as a user would. Maybe there is some element that is at the bottom that triggers the next page load you would need to scroll to that spot and force the next load. Repeat.
The easier thing to do would be to first check if you can use their api first if it is exposed in some way even if you have to login with authentication and use the api to page.
If you for some reason you cannot because the data is rendered from multiple apis you would need to potentially call more JavaScript calls on the page. Basically simulating scrolling as a user would. Maybe there is some element that is at the bottom that triggers the next page load you would need to scroll to that spot and force the next load. Repeat.
Thanks for the quick response!
Yeah, that could work - I found which element triggers the loading but how can I possibly simulate scrolling?
Let's take this conversation off here and reach out to me