Tools of Extracting Text from Webpage

#python #opensource #devops #api

Hey Dev Community,

Recently, I wrote a blog for sharing some of my found tools for scraping text from webpage.

This included free open source libraries and cloud API.

Have a look to know which tools suits your need and maybe give a try ?

Let me know if I've missed any popular resource. I'll add that in my next blog.

Top comments (2)

Great to see your post :)

Here's a failure scenario. Please try to extract the Trip Advisor URL. Ex: TripAdvisor_Tourism

Suggestion -

It would be great to let the users specify the structured schema for the data extraction.
Text extraction is truly a basic thing, However the value that you could provide is with the structured data extraction.
Ability to auto web crawl and handle all the related link extraction. Basically a web spider kind of thing.
Good to mention the known issues or challenges such as rate limits, how are you going to handle the 3rd party content crawl issues or rate limits?
Please mention various use-cases or scenarios where this API will be helpful. I see you have mentioned few statements in the conclusion, however I doubt it will be super useful to the end users unless the structured data is provided
Think in terms of a business user. How would these APIs will benefit from various angles needs to be thought well.

Thanks, Ranjan for the feedback. I've shared this internally with the team and will let you know once we address it.