DEV Community

zvone187
zvone187

Posted on

45 ways to break an API server (negative tests with examples)

#webdev #api #testing #javascript

As developers, we strive to write error-free code, but no one actually does so because...well, bugs. In order to catch those pesky bugs before they wreak havoc on our applications, we rely on automated testing. While positive tests ensure our code works as intended, negative tests play a crucial role in validating that our applications are robust enough to handle unexpected input and edge cases.

I'm working on Pythagora, an open source tool that writes automated integration tests by itself (well, with a bit of help from GPT-4) without you, the dev, having to write a single line of code. Basically, you can get from 0 to 80% code code coverage within 30 minutes (video).

We just created a feature that automatically generates negative tests from the entire test suite with a single command. While building that feature, I researched what are different ways one can break an API server to test if it handles errors gracefully so here it is - a comprehensive list of ways with which you can break your server, if it doesn't handle errors properly.

1. Empty or missing required fields 

{
    "endpoint": "/api/users",
    "body": {
        "username": "",
        "email": ""
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

2. Invalid field values - exceeding character limits 

{
    "endpoint": "/api/users",
    "body": {
        "username": "ThisIsAnIncrediblyLongUsernameThatExceedsTheCharacterLimit"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

3. Invalid field values - malformed data 

{
    "endpoint": "/api/users",
    "body": {
        "email": "invalid-email@"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

4. Extra or irrelevant keys in the payload 

{
    "endpoint": "/api/users",
    "body": {
        "username": "validuser",
        "extra_key": "irrelevant_value"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

5. Incorrect or invalid HTTP methods 

{
    "endpoint": "/api/users/123",
    "body": {},
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

6. Invalid endpoint paths 

{
    "endpoint": "/api/nonexistent_endpoint",
    "body": {},
    "method": "GET"
}
Enter fullscreen mode Exit fullscreen mode

7. Query parameters instead of using the request body in POST requests 

{
    "endpoint": "/api/users?username=testuser",
    "body": {},
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

8. Missing or invalid authentication headers (e.g., API keys) 

{
    "endpoint": "/api/users",
    "body": {},
    "method": "GET",
    "headers": {
        "Authorization": "Invalid API_KEY"
    }
}
Enter fullscreen mode Exit fullscreen mode

9. Incorrect data structure - array instead of an object 

{
    "endpoint": "/api/users",
    "body": [
        "username": "testuser",
        "email": "test@example.com"
    ],
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

10. Incorrect data structure - object instead of an array 

{
    "endpoint": "/api/users",
    "body": {
        "users": {
            "username": "testuser",
            "email": "test@example.com"
        }
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

11. JSON formatting issues - invalid Unicode characters 

{
    "endpoint": "/api/users",
    "body": {
        "username": "test\uFFFFuser"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

12. Duplicate keys in the payload 

{
    "endpoint": "/api/users",
    "body": {
        "username": "testuser",
        "username": "duplicate"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

13. Invalid or unsupported content types (e.g., sending XML instead of JSON) 

{
    "endpoint": "/api/users",
    "body": "<user><username>testuser</username><email>test@example.com</email></user>",
    "method": "POST",
    "headers": {
        "Content-Type": "application/xml"
    }
}
Enter fullscreen mode Exit fullscreen mode

14. Exceeding payload size limits 

{
    "endpoint": "/api/users",
    "body": {
        "large_data": "A very large data string that exceeds the server's payload size limit..."
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

15. Invalid or expired authentication tokens 

{
    "endpoint": "/api/users",
    "body": {},
    "method": "GET",
    "headers": {
        "Authorization": "Bearer expired_token"
    }
}
Enter fullscreen mode Exit fullscreen mode

16. Using special characters in field values 

{
    "endpoint": "/api/users",
    "body": {
        "username": "test!@#$%^&*()-user"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

17. Sending nested objects instead of simple key-value pairs 

{
    "endpoint": "/api/users",
    "body": {
        "user": {
            "username": "testuser",
            "email": "test@example.com"
        }
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

18. Sending data in the wrong data type (e.g., string instead of integer) 

{
    "endpoint": "/api/users",
    "body": {
        "age": "25"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

19. Sending null values for required fields 

{
    "endpoint": "/api/users",
    "body": {
        "username": null
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

20. Using reserved keywords in field names 

{
    "endpoint": "/api/users",
    "body": {
        "class": "user"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

21. Sending incomplete or malformed multipart file uploads 

{
    "endpoint": "/api/upload",
    "body": {
        "file": "incomplete_file_data"
    },
    "method": "POST",
    "headers": {
        "Content-Type": "multipart/form-data"
    }
}
Enter fullscreen mode Exit fullscreen mode

22. Incorrect or missing URL encoding for special characters 

{
    "endpoint": "/api/users?username=test user",
    "body": {},
    "method": "GET"
}
Enter fullscreen mode Exit fullscreen mode

23. Sending the request body in GET requests 

{
    "endpoint": "/api/users",
    "body": {
        "username": "testuser"
    },
    "method": "GET"
}
Enter fullscreen mode Exit fullscreen mode

24. Invalid date or time formats 

{
    "endpoint": "/api/users",
    "body": {
        "birthdate": "01-25-1990"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

25. Using non-ASCII characters in field names 

{
    "endpoint": "/api/users",
    "body": {
        "üsername": "testuser"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

26. Sending deeply nested objects 

{
    "endpoint": "/api/users",
    "body": {
        "user": {
            "profile": {
                "details": {
                    "nested": "too_deep"
                }
            }
        }
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

27. Using non-printable or control characters in field values 

{
    "endpoint": "/api/users",
    "body": {
        "username": "test\u0008user"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

28. Sending the same field multiple times with different values 

{
    "endpoint": "/api/users",
    "body": {
        "username": "testuser",
        "username": "different"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

29. Missing or invalid content-length headers for request bodies 

{
    "endpoint": "/api/users",
    "body": {
        "username": "testuser"
    },
    "method": "POST",
    "headers": {
        "Content-Length": "invalid"
    }
}
Enter fullscreen mode Exit fullscreen mode

30. Using spaces or special characters in field names 

{
    "endpoint": "/api/users",
    "body": {
        "user name": "testuser"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

31. Sending invalid or malformed JSONP callbacks 

{
    "endpoint": "/api/users?callback=invalid(callback)",
    "body": {},
    "method": "GET"
}
Enter fullscreen mode Exit fullscreen mode

32. Sending the payload as a single string instead of key-value pairs 

{
    "endpoint": "/api/users",
    "body": "username=testuser&email=test@example.com",
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

33. Sending boolean values as strings (e.g., "true" instead of true) 

{
    "endpoint": "/api/users",
    "body": {
        "active": "true"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

34. Using non-standard HTTP methods (e.g., PATCH, CONNECT) 

{
    "endpoint": "/api/users/123",
    "body": {
        "username": "updateduser"
    },
    "method": "PATCH"
}
Enter fullscreen mode Exit fullscreen mode

35. Sending unsupported HTTP version numbers 

{
    "endpoint": "/api/users",
    "body": {},
    "method": "GET",
    "httpVersion": "HTTP/3.0"
}
Enter fullscreen mode Exit fullscreen mode

36. Sending multiple authentication headers (e.g., both API key and token) 

{
    "endpoint": "/api/users",
    "body": {},
    "method": "GET",
    "headers": {
        "Authorization": "Bearer token_value",
        "API-Key": "api_key_value"
    }
}
Enter fullscreen mode Exit fullscreen mode

37. Sending unnecessary or invalid CORS headers 

{
    "endpoint": "/api/users",
    "body": {},
    "method": "GET",
    "headers": {
        "Access-Control-Allow-Origin": "*"
    }
}
Enter fullscreen mode Exit fullscreen mode

38. Sending conflicting query parameters and request body data 

{
    "endpoint": "/api/users?username=testuser",
    "body": {
        "username": "different_user"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

39. Using non-standard characters in authentication header values 

{
    "endpoint": "/api/users",
    "body": {},
    "method": "GET",
    "headers": {
        "Authorization": "Bearer t@ken_value"
    }
}
Enter fullscreen mode Exit fullscreen mode

40. Sending negative numbers for fields that should only accept positive values 

{
    "endpoint": "/api/users",
    "body": {
        "age": -25
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

41. Sending timestamps in the future or past beyond expected range 

{
    "endpoint": "/api/users",
    "body": {
        "birthdate": "01-25-1800"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

42. Using HTML, JavaScript, or SQL code in field values to attempt code injection 

{
    "endpoint": "/api/users",
    "body": {
        "username": "<script>alert('test')</script>"
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

43. Using different character encodings in the payload (e.g., UTF-8, UTF-16) 

{
    "endpoint": "/api/users",
    "body": {
        "username": "téstuser"
    },
    "method": "POST",
    "headers": {
        "Content-Type": "application/json; charset=UTF-16"
    }
}
Enter fullscreen mode Exit fullscreen mode

44. Sending arrays with mixed data types 

{
    "endpoint": "/api/users",
    "body": {
        "values": [1, "string", true]
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

45. Sending field values as arrays or objects instead of simple data types (e.g., string, number) 

{
    "endpoint": "/api/users",
    "body": {
        "username": ["testuser"]
    },
    "method": "POST"
}
Enter fullscreen mode Exit fullscreen mode

That's it. I hope this list gave you new ideas to test and protect your server.

If you found this post valuable, it would mean the world to me if you could support us by starring Pythagora Github repo.

And, if you try it out, please let us know your feedback, we're happy to hear it.

Top comments (16)

Collapse
 
michaeltharrington profile image
Michael Tharrington •

Wow! This is super cool. Appreciate ya sharing all this research. Pythagora sounds like a really cool tool!

Collapse
 
zvone187 profile image
zvone187 •

Thank you so much Michael! Did you maybe try Pythagora? I'd love to hear your feedback.

Collapse
 
michaeltharrington profile image
Michael Tharrington •

To be honest, I haven't.

I'm a Community Manager here with admittedly very limited technical abilities. 😅 That said, I do feel like I have a decent understanding of what's going on here — you've created Pythagora to automatically generates negative tests to make sure that whatever is being built can handle unexpected inputs and edge cases. And in the process of doing that ya researched a bunch of ways one can break an API server, which is what ya have shared here.

So yeah, no use from me here (yet! ... hey, there's always a chance I get more techie in the future), but I do really think this is a great idea!

Thread Thread
 
zvone187 profile image
zvone187 •

Ah, got it. Btw, it's great that you understood what's happening here without being a QA or a dev. I'd say this is advanced for a non-tech person. Btw, do you have any devs around you who can try out Pythagora? As you can imagine, we're super early so any support/feedback would mean the world to me.

Thread Thread
 
michaeltharrington profile image
Michael Tharrington •

Thanks a bunch, ya explained it well which helps!

I feel you on the struggle of starting small and trying to reach folks. So, while I can't commit on behalf of others, I will definitely throw the suggestion out there to our devs on the team via Slack and see if anybody is interested. 🙂

And now I'm gonna put my DEV Community Manager hat on for a sec... One thing you might consider with Pythagora is setting up a free organization on DEV for it. You can check out our organization info page to learn about some of the perks of using an org and if you wanna get a sense of what it's like, you can see the DEV Team org page here — just click through some of our posts and you'll see how our org branding appears around them. Using orgs helps with general brand recognition because as folks are reading your posts and learning about Pythagora, they'll also see your logo in places and you have the option to point them to your website/repo/etc. I'll not blab about orgs for too much longer, haha, you can check out the info page and feel free to hit me up if you have any questions!

I really wish you and the rest of the Pythagora team the best! It really does sound like y'all are building something cool.

Thread Thread
 
zvone187 profile image
zvone187 •

Oh nice, did it! Thanks for this, I didn't know dev.to has orgs.

Re sharing Pythagora, thank you so much!!! This really means a lot. Wherever you can share it, I'd be more than thankful.

Collapse
 
artdevgame profile image
Mike Holloway •

Cool article, simple & clear.

@zvone187 Reading the comments it looks like you're keen for feedback & even though I haven't tried it yet (I will at some point), my first thoughts are that the name Pythagora is an interesting choice given that the tool generates tests "for your Node.js app"

If I didn't have the context that this article provides, I might have overlooked the repo if I had discovered it another way because I thought it was a tool for a Python codebase.

Collapse
 
zvone187 profile image
zvone187 •

Hi Mike,

Thanks! I'm glad you like it.

Re name, yes, you're completely right. I didn't think too long when I was choosing the name - I just took what had a domain available. It will come to it's place when we support Python.

Btw, yes, I'm definitely eager to hear feedback so I'd appreciate so much to hear yours as well. Also, if you get stuck anywhere, let me know, I'd be happy to help.

Collapse
 
starboysharma profile image
Pankaj Sharma •

I think most of this type of problem can be easily managed by joi package.

Collapse
 
zvone187 profile image
zvone187 •

Oh nice, good one. Never used it but it seems like it should handle many of these. Btw, were you able to see Pythagora (github.com/Pythagora-io/pythagora)? Since you're familiar with automated testing, I'm curious to hear your opinion on it. Would you use generated tests for your projects?

Collapse
 
starboysharma profile image
Pankaj Sharma • • Edited

Pythagora sounds interesting will going to take a look soon.

Thanks for sharing 🙂

Thread Thread
 
zvone187 profile image
zvone187 •

Oh awesome, I'm looking forward to hearing what you think.

Collapse
 
rmaurodev profile image
Ricardo •

Excellent !

Collapse
 
zvone187 profile image
zvone187 •

Thanks Ricardo! Are you used to building negative tests in your team?

Collapse
 
manuelbrs profile image
Juan Manuel Bello •

Great Articule,

thanks for sharing, good cases to keep in mind.

Collapse
 
zvone187 profile image
zvone187 •

Thank you so much Juan!

View full discussion (16 comments)

Read next

rustcodeweb profile image

jQuery Tutorials

Rustcode -

18xdeveloper profile image

News admin dashboard

Sheikh Abdur Rohit -

devshi profile image

Creating Your First Backend with Node.js: Step-by-Step Guide in 2024

Devshi Bambhaniya -

a727283 profile image

Quick introduction to the Prisma ORM

Awais -