Internet and The Web

The Internet is very vast and what we use on a daily basis, is just a chunk of it. The Internet is much more than that but first, we are going to take a quick look at what is the difference between the Internet and the Web.

Internet: The Internet is a collection of various smaller networks where each node can be a server, a laptop, a pc, a smartphone e.tc.

Web: In the earlier days of the Internet, data was used to transfer across the Internet or the network but no Web existed at that time, but in 1989, Tim Berners-Lee introduced a web called World Wide Web (WWW) which can be used to access hyperlinked text or web pages all across the Internet. So basically, the Web is a piece of software that allows us to access hyperlinked text or web pages over the Internet.

Parts of the Web

But the web doesn't only include sites like Facebook, Instagram, Google, etc. The websites or the part of the web which we are easily able to surf is just 4-6% of the whole composition of the web. There are parts of the web. Which are as follows:

• Surface Web
• Deep Web
• Dark Web

Surface Web

The Surface Web (also called the Visible Web, Indexed Web, Indexable Web or Lightnet) is the portion of the World Wide Web that is readily available to the general public and searchable with standard web search engines. It is the opposite of the deep web, the part of the web indexed by web search engines. It is the part of the web which we can access directly via the browser, like Google, Instagram, Facebook, and many of the other sites which you may visit. As we can see from the image above the Surface Web only consists of 4-6% of the Information available on the Internet. However, the composition is not fixed to 4 or 6%, It varies from time to time. The Surface Web is made with a collection of public Web pages on a server accessible by any search engine. Basically, It is the part of the Internet that we can access normally using any web browser, like Facebook, Instagram, Twitter, and any of our favorite sites.

Deep Web

The Deep Web, invisible web, or hidden web are the parts of the Web that are not indexed by any search engines. The contents of the deep web are hidden behind login forms, and includes uses such as webmail, online banking, restricted access social-media pages and profiles, some web forums and code language that require registration and authorization for viewing contents, and paywalled services such as video on demand and some online magazines and newspapers. On the deep web we can find an awesome list of books and other types of resources for everything ranging from education to entertainment and useful to harmful.

So, as we know that the contents of the deep web are not indexed by any search engines, then how can we access such contents, if we are not able to find it on any search engines?

The Contents of the deep web can be located and accessed by a direct URL or the IP address but may require a password or any kind of authorization or authentication or other security access to get past public pages.

Indexing

Methods that prevent web pages from being indexed by traditional search engines may be categorized as one or more of the following:

• Contextual web: pages with content varying for different access contexts (e.g., ranges of client IP addresses or previous navigation sequence).
• Dynamic Content: dynamic pages which are generated by any scripting (i.e. generated by PHP or JS)
• Limited Access Content: Pages that use authentication or authorization to verify if a user is allowed to access the specific page or not (i.e banking applications, applications with paid subscriptions, web mails, etc.)
• Non-HTML / Text Content: textual content encoded in multimedia (i.e images or videos) files or specific file formats that are not handled by the search engines.
• Private Web: Sites or applications that require registration or login (password-protected contents) i.e. pages of social media sites like Facebook, and Instagram, Pages of Banking applications, admin panel of applications
• Scripted Content: pages that are only accessible through links produced by JavaScript as well as contents dynamically downloaded from Web Servers via Flash or Ajax solutions
• Software: certain content is intentionally hidden from the regular Internet, accessible only with special software, Tor, I2P, Tails, or other darknet software. For example, Tor allows users to access websites using the .onion server address anonymously, hiding their real IP address.
• Unlinked Content: pages that are not linked by other pages, which may prevent web crawling programs from accessing the contents. This content is referred to as pages without backlinks (also known as inlinks). Also, search engines do not always detect all the backlinks of the search or indexed web pages.
• Web Archives: Web archival services such as Wayback Machine enables the user to access the archived version of the web across time, including websites that are no longer accessible and are not indexed by search engines like Google. > The Wayback Machine may be called a program for viewing the deep web, as the web archives that are not from the present cannot be indexed by search engines, as past versions of websites are impossible to view through a search. All websites are updated at some point, which is why web archives are considered as Deep Web Content. {: .prompt-info }

Now, most of the people wonders that, Is The Deep Web illegal to surf or visit?

So answering that question, the answer is No, It's not. Surfing or visiting the Deep Web or even the Dark Web is not illegal in most of the places. It depends on the place from where you are accessing it. As we know China has banned the most famous sites from the surface web like Google, Facebook, YouTube, etc. So, for the people living in China or let's say Koria, Yes, It's illegal, even surfing on the surface web sites mentioned above. So it depends but for most the places, It is not illegal to surf or visit the deep web. Unless you do something illegal on that. I mean if you are surfing the deep web or the dark web out of curiosity or for educational purposes. It's perfectly fine but, If you're trying to do anything bad or things that are prohibited to do on the Internet. yes, it will be illegal then. So it totally depends on you.

Enough talking about the deep web, now let's explore what the dark web is.

Dark Web

The Dark Web is the World Wide Web content that exists on darknets: overlay networks that use the Internet but requires specific software, configurations, or authorization to access. Through the dark web private computer networks can communicate and conduct business anonymously without disclosing identifying information, such as the user's location. The dark web forms a small part of the deep web, The part of the web which is not indexed by any search engines, although sometimes the term deep web is mistakenly used to refer specifically to the term dark web.

The darknet which constitutes the dark web includes small, friend-to-friend peer-to-peer networks, as well as large, popular networks such as Tor, FreeNet, I2P and Riffle operated by public organizations and individuals. Users of the dark web refer to the regular web (surface web websites) as Clearnet due to its unencrypted nature. The Tor dark web or onionland or darknet uses the traffic anonymization technique of onion routing under the network's top-level domain with the suffix .onion or In other words, the URL or the Domain of the dark web websites uses .onion as suffix instead of .com, .org, .in, .xyz suffix used in surface web or Clearnet websites.

More on Dark Web

The dark web is often confused with the term deep web, the parts of the web which are not indexed by the search engines. The dark web is a small part of the deep web, both the deep web nor the dark web is not indexed by any search engines but we can surf or visit the deep web with normal browsers, and maybe in some cases if the websites we are trying to surf is banned in your area, you may need to use any VPN (Virtual Private Network) or any proxy in order to access them, but in the case of dark web, it requires specific software, configuration or authorization to access it. The term dark web is first emerged in 2009, however, It is unknown when it dark web first emerged. Don't get confused here, the term dark web was emerged in 2009. but no one knows from when the dark web is existing. The confusion of the terms deep web and the dark web dates back to at least 2009. Since then, especially in reporting on Silk Road (Silk Road was an online black market, and the first modern darknet market.), these two terms were conflated (combined) despite the recommendations that they should be distinguished.

The dark web also known as the darknet websites, are accessible only through networks such as Tor ('The Onion Routing' project) that are specially created for the dark web. Tor-accessible are widely used among the darknet users and can be identified by the domain ".onion"

For those who are wondering what is available on the dark web, what are its contents?

Contents of the Dark Web

• Ransomware: The dark web is also used in certain extortion-related processes. Indeed, It is common to observe data from ransomware attacks on several dark web sites (i.e data sales sites, public repository sites)
• Botnets: Botnets are often structured with their command-and-control or C2 servers based on censorship-resistant hidden service, creating a large amount of bot-related traffic.
• Darknet Markets: Darknet markets where you can buy illegal items such as Arms, Ammos, Grenades, even Rocket Launchers, etc.
• Bitcoin Services: Bitcoin Services like buying them at cheap prices or selling them etc. dark web heavily utilizes bitcoin as their main currency as bitcoin transactions are not stored anywhere.
• Hacking Groups and Services: Hacking services like, hiring hackers for doing any illegal tasks, also you can meet many hackers there but may or may not be good or bad.
• Financing and Fraud: You can buy Credit cards, Currencies and can also get scammed as there are many fraudsters are grooming around the dark web.
• Illegal Pornography: You can find illegal pornography content such as leaked videos or child pornography content, also you can buy them on the darknet markets.
• Social Media: There are also social media available on the dark web. and also Facebook has its own version of dark web social media if I'm not mistaken.
• Hoaxes and Unverified Contents: As it is anonymous, people can post whatever they want without taking the risk of any type of action against them, so there are videos of people murdering anyone, and torturing someone. basically, all types of uncensored content.
• Lots of Information: Despite the fact that there are bad things on the dark web there are also some good things like you can find information about literally anything you can think of. From educational to destructional, like you can find even books about how to mark bombs and such things.

Now one question still arrives, Is it illegal to surf or visit the dark web?

As answered previously, No, It's not, it depends on what you do. Let's take an example you have a knife, you can use it to cut vegetables, use it in the kitchen, and other kinds of stuff but as you have the knife in your hand, you can also use it to hurt someone or murder someone. So let's assume the dark web or deep web is the knife, you can either use it for your own gain, for informational or educational purposes or you can use it to do some really bad or illegal stuffs.

So, the thing is the knife (in our case, the dark web or the deep web) is not illegal but rather the actions you take with it are legal or illegal.

Another question, Is it safe to surf the dark web or deep web?

This question does not have any straight answer, I mean, It is and at the same time It's not. Like there are lots of scammers, black-hat hackers, and fraudsters are grooming around them. So, again it's your actions that will make you safe and unsafe. like talking in that sense, The surface web or Clearnet is also not safe, like there are so many scammers, hackers and fraudsters are on the surface web. as the majority of the Internet users are on the surface web, so the hackers, scammers, and other bad threat actors are more active on surface web. So, again it's your consciousness and actions, which makes you safe and unsafe.