DEV Community

Crawling Websites in React-Native

K on October 16, 2017

Coming from years of web developing React-Native feels like a fresh start to me. You get better access to native functionality AND you have fewer r...

Read full post

K • Jun 16 '22

Glad this article is still helpful after all that time :D

Acaraccioli • May 13 '20

Hello K, great post learned a lot I didnt know this was possible using fetch. Just one quick question. How would you manage if you wanted to fetch some quick data in front end (react) but had to enter information in an input tag and maybe even click a button? I hope you can help me out a bit

K • May 14 '20

Glad you liked it.

I'd use React hooks.

function MyComponent(props) {
  const [info, setInfo] = React.useState("");
  const [remoteData, setRemoteData] = React.useState(
    "No data fetched yet!"
  );

  async function load() {
    const response = await fetch(
      "http://example.com?info=" + info
    );
    const text = await response.text();
    setRemoteData(text);
  }

  return (
    <div>
      <input
        value={info}
        onChange={(e) => setInfo(e.target.value)}
      />
      <button onClick={load}>Fetch</button>
      <textarea>{remoteData}</textarea>
    </div>
  );
}

Acaraccioli • May 14 '20

That is great I think that might works thanks a lot! I'm just trying to figure out this error:
Access to fetch at 'MyUrl' from origin 'localhost:8100' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's. I've found to fix it adding {mode:"no-cors"} in the fetch call but the object returns null. Do you know anything about this kind of error?

Hardiansa • Mar 1 '19

Hi K, this awesome post.
So already try and it's working fine,

But I have some problem,
I try scrape from streaming service website (anime).

But there are no video tag inside the website.

So I try to inspect element, then I saw that website need to click "play" button then I got ifarame embedded html document with video tag.

So how can i do click on cheerio then get embeded video?

Thanks

K • Mar 1 '19

Sorry, I don't know if Cheerio works with JavaScript sites.

One way to solve this would be to check if you could calculate the video URL from the data that is already in the HTML.

Otherwise I don't know.

Bagustyo • Nov 14 '18

why Async ? what if just fetch ?

K • Nov 14 '18

You can use fetch without async/await. React-Native supports async/await, that's why I used it, but it isn't needed, you can use promises directly :)

Bart Karalus • Oct 16 '17

Nice one! I had no clue there was a jquery-like tool for RN. Very useful.

Crawlbase • Mar 21

Thankyou! Impressive breakdown of web crawling in React-Native! Your methodical approach and clear explanations make it accessible for anyone diving into this field. Don't forget to streamline your efforts with Crawlbase for enhanced efficiency.

Martin Sone • Aug 18 '18

Hi K, could the above crawling applies to reactjs or only to react native?

K • Aug 18 '18

Only React-Native, because you can't access other websites from within a browsers, just sites from the same domain or such that are CORS enabled.

ynstl • May 31 '19

No, it's not working. ERROR.
ESLint Parsing error: Unexpected token

Prakort Lean • Aug 12 '19

Please go in depth, i couldn't get cheerio-without-node-native to work

Richard Joseph • Jan 27 '21

There's also react-native-cheerio, I've not yet used it myself but, obviously, I'm doing the research, also.

lilraja-x • Dec 25 '22

I'm new to this react native thing... I've tried your exact method as of now yet nothing is displayed on my react native mobile app.
I'm new to this so your help will matter alot.

lilraja-x • Dec 25 '22

There's no error but also nothing's displayed.