How to fix the "Element not found" error in Selenium

I work as a Software Engineer at Endtest.

In this article, I will show you how to deal with the most common error in Selenium.

Why is it happening?

When you instruct Selenium to look for an element, you provide a locator.

A locator can be one of the following:
• ID
• Name
• Class Name
• XPath
• CSS Selector
• Tag Name
• Partial Link Text
• Link Text

If Selenium returns an error saying that it could not find the element, it doesn't mean that Selenium is broken or not good enough for your super-dynamic web application.

You just have to investigate a bit.

Possible causes and solutions

1. You did not configure the explicit wait.

The Explicit Wait basically tells Selenium how much to wait for an element to appear before giving up and throwing the "Element not found" error.

If you configure an Explicit Wait of 20 seconds, Selenium will query the DOM for your element once every 100 milliseconds for 20 seconds.

If Selenium finds your element in 1 second, it will interact with it and it will move on to the next step, it won't wait another 19 seconds.

element = WebDriverWait(driver, 20).until(
        EC.presence_of_element_located((By.ID, "submit"))

This gives you the flexibility to deal with asynchronous requests and unpredictable loading times.

Not properly configuring the Explicit Wait is why you'll hear some people say "Oh, I tried Selenium, it doesn't work, my tests are too flaky".

If you ever find that claim in an article, you should unfollow the author.

In Endtest, you can configure the Element Load Timeout for each test suite, this means that you don't have to add a wait before each step.

You can find more details about that in this chapter.

2. You are using a dynamic locator.

A dynamic locator is an attribute of an element that does not keep the same value over time.

That locator can change when you refresh the page, when a new session starts or when a new build is deployed.

Dynamic locators can be easily identified.

They usually have some random pattern of letters and/or numbers.

The input element has the following ID: ember596.

After we refresh the page, that ID will have a different value:

Using a dynamic ID as a locator for your Selenium tests is obviously a bad idea.

If we take a closer look at that element, we can see that it has a stable Name attribute.

So, the solution would be to use the Name attribute as the locator:

username_input = driver.find_element_by_name("username")
username_input.send_keys("walter_white");

If we don't have any stable attributes, we can just fetch the Full XPath.

Notice that I wrote Full XPath, and not XPath.

If we would get the XPath like this:

We would end up with the following XPath:

//*[@id="ember568"]

That XPath is actually using the same dynamic ID, which means it's not reliable.

But if we would get the Full XPath like this:

We would end up with the following XPath:

/html/body/div[1]/div[1]/div/div/div[2]/div/form/div[1]/input

And that XPath would be reliable, since it does not not contain any dynamic attributes.

If you don't want to rely on such an XPath, you can always write your own CSS Selectors.

I actually wrote a detailed article about that.

In Endtest, we have a smarter way of dealing with dynamic locators and you can read about it here.

3. Your element is inside an iframe.

An iframe is used to embed another document within the current HTML document.

When you look at a page without inspecting the elements, you can't really tell if an element is inside an iframe or not.

Let's take a look at this element:

It has the ID "cars", we try to use that ID in Selenium:

cars_select = driver.find_element_by_id("cars")
cars_select.select_by_visible_text("Volvo")

And we get an "Element not found" error.

If we would take a closer look in the DOM tree, we can see that the element is actually inside an iframe:

All we have to do is use tell Selenium to switch to that iframe and then switch back after we're done interacting with the elements from that iframe:

the_iframe = driver.find_element_by_id("iframeResult")
driver.switch_to.frame(the_iframe)
cars_select = driver.find_element_by_id("cars")
cars_select.select_by_visible_text("Volvo")
driver.switch_to.default_content()

Endtest can automatically deal with iframes and multiple tabs, you can see how it does that in this short video.

4. Your element is in a different browser tab.

There are plenty of scenarios where we use more than one browser tab in a user journey.

For example, let's say we have some Social Media links in the footer of our website.

When we click on them, those links will open in new browser tabs.

Another example would be a Single Sign-On flow.

Because your browser automatically switches focus to the newly opened tab, you might be inclined to believe that Selenium does the same.

But it doesn't.

You have to tell Selenium to switch its focus to a different browser tab.

currentTab = 0
nextTab = currentTab + 1
driver.switch_to_window(driver.window_handles[nextTab])
currentTab = nextTab

You can switch back and forth between browser tabs, just make sure you're keeping track of which tab you're on.

P.S. It worries me that "modern" testing frameworks like Cypress do not have the functionality to switch between browser tabs. This is a major limitation in their architecture.

5. Your element is in a Shadow DOM.

Shadow DOM allows hidden DOM trees to be attached to elements in the regular DOM tree.

Similar to iframes, you can't really tell if your element is in a Shadow DOM until you inspect it.

A good example can be found in the Extensions page from your Chrome browser.

This is the "Search extensions" input:

If we would try to find it with Selenium:

driver.find_element_by_id("searchInput")

We would get an "Element not found" error.

But if we would take a closer look in the DOM, we can see that it's located in a Shadow DOM:

So, our Selenium code should actually look like this:

shadow_element = driver.find_element_by_id("search")
shadow_root = driver.execute_script('return arguments[0].shadowRoot', shadow_element)
main_driver = driver
driver = shadow_root
driver.find_element_by_id("searchInput")

6. Is your element really there?

It never hurts to check.

The easiest way to debug is to add a step that takes a screenshot.

Always check the screenshot and see if your element is actually there, maybe you forgot to include your Login steps or something like that.

You can also fetch the page source to check there:

the_page_source = driver.page_source

You can also add video recording to your Selenium tests and I wrote a detailed article on how to do that.

7. Isn't it a different error?

You should always read the error message, maybe Selenium is actually telling you that your element is not clickable, not that it can't find it.

The "Element is not clickable" or "Element is not interactable" errors can be fixed by finding out what covers your target element.

Maybe it's a popup or maybe it's a fixed footer and you haven't scrolled down enough.

The good news is that Selenium always tells you in the error message what element is covering your target element.

Conclusion

Selenium resembles a car with manual transmission.

If you don't know how to drive it, you won't get very far.

If someone ever tells you that Selenium tests are flaky, point them to this article.

I added a few references to Endtest because it also uses a Selenium engine to locate elements in web applications.

As mentioned at the beginning of this article, I work there and this allowed me to become an expert in Selenium.

So, if you have any technical questions, just use the Comments section below.