DEV Community

Discussion on: Scrape ResearchGate all institution members in Python

 
dmitryzub profile image
Dmitriy Zub ☀️ • Edited

A possible workaround is to run the script in terminal, where data will be saved to the file and then load data inside the notebook.

Not very convenient though.

Thread Thread
 
dmitryzub profile image
Dmitriy Zub ☀️

Let me know if you figure out it or not 🙂

Thread Thread
 
datum_geek profile image
Mohamed Hachaichi 🇺🇦

It does, it works using the terminal (ps, the code inside VS code also does not work, the loop does not break at all :p ).

Dimi, tell me if you could manage to collect extra data on ResearchGate (inside of each profile page)

Thread Thread
 
dmitryzub profile image
Dmitriy Zub ☀️

Awesome 👍 What do you mean that it doesn't work in vscode? The code has been written inside vscode. How do you run the code inside vscode? Using code runner extension or from the terminal?

I'm have planned a blog post about a scraping profiles data. I'll try to publish it next week.

Thread Thread
 
dmitryzub profile image
Dmitriy Zub ☀️ • Edited

@datum_geek, the blog post about scraping profile information is available: dev.to/dmitryzub/scrape-researchga...

Thank you for your suggestion :)