Fantastic Programming Project Ideas and Where to Find Them (a beginner-friendly version)

20 cool project ideas for app development, web development, Data Science and Machine Learning and some goldmines of cool project ideas

We all know that working on personal projects is a really important part of learning. Its honing the discipline. Moving from theory to practice. A way of learning by doing.

But isn't the task of getting good project ideas the one that also prevents you from building stuff, in the first place? Isn't that a narrow bottleneck? Aren't you tired of those lists of programming projects ideas that suggest you to build things like a To-Do list app, a snake game, a calculator, an ecommerce website or something else that no one is going to find cool?

I find them really boring because no one is ever going to be excited about using the final result. To be honest, not even me.

I believe that you can train your mind to get better ideas - anyone can think of good ideas. If you look at enough such projects and maybe work on a few, your mind will learn to recognise cool things that will be interesting to work on.

So, here I present some project ideas that I find really cool, grouped by their sources - my "goldmines" of fantastic project ideas.

Source #1. Devpost.com

Browsing through other people's hackathon projects on devpost.com is a great way to come across good project ideas because -

• A lot of them are just small, cozy, warm pet projects: being personal projects built by young programmers in just some 12 or 24 or 48 hours, they are definitely doable (Yes, you can do it!)
• You can know that you are building something worthwhile: most of them are hackathon winning projects

How cool would it be if you created: -

1. A web application that lets you search inside a YouTube video and gets you to the point where that word is uttered - a Ctrl-F capability for videos.
And tricked yourself to learn - Python, Basic web development (HTML/CSS, Javascript)

2. A browser extension that referred you to a story with an opposite political view as the one that you are reading. (This will combat the effects of newsfeeds that allow people to only see posts on social media and news sites that agree with their point of view)
And tricked yourself to learn - Basic web development (HTML/CSS, Javascript, jQuery), maybe some Machine Learning

3. A web app that allows you to enter the day and place to which you plan to travel and the amount of money you will be bringing along, and will provide you useful information about weather conditions and the value of your money.
And tricked yourself to learn - Basic web development (HTML/CSS/Javascript), APIs

4. A simple notifications app that lets you select messaging apps that you want to block notifications from when the frequency of notifications exceeds 1 every 3 seconds.
And tricked yourself to learn - Android development

5. A messaging app that automatically sends a text to your loved ones letting them know that you've reached a particular destination because you often forget to do so.
And tricked yourself to learn - Android app development

Source #2: Kaggle

I believe that if you want to get into Data Science/ML, Kaggle is your one-stop shop to learn and practice the craft:

• Datasets: With around 300 competition challenges, all accompanied by their public datasets, and 9500+ datasets in total (and more being added constantly) this place is like a treasure trove of Data Science/ ML project ideas.

• Kernels: All the datasets have a public kernels tab where people can post their analysis for the benefit of the entire community. So, anytime you feel like you don’t know what to do next, you can be sure to get some ideas by looking at those kernels. Besides, a lot of those kernels are written especially to help the beginners.

• Courses: This tab contains free, practical, hands-on courses that cover the minimum prerequisites needed to quickly get started in the field. The best thing about them? — everything is done using Kaggle’s kernels (described above). This means that you can interact and learn.. no more passive reading through hours of learning material!

So, how would you like to work on a project that analyses:

6: Spotify's Worldwide Daily Song dataset,

This dataset contains the daily ranking of the 200 most listened songs in 53 countries from 2017 and 2018 by Spotify users. It contains more than 2 million rows, which comprises 6629 artists, 18598 songs for a total count of one hundred five billion streams.

And find answers to:

• How long do songs "resist" on the top 3, 5, 10, 20 ranking in your country? Which songs are the outliers?
• Which countries have similar tastes in music?
• How long time does a top ranking song takes to get into the ranking of neighbor countries?

7: Young people survey dataset that explores the preferences, interests, habits, opinions, and fears of young people.

1010 students were asked questions regarding their:

• Music preferences
• Movie preferences
• Hobbies & interests
• Phobias
• Health habits
• Personality traits, views on life, & opinions
• Spending habits
• Demographics

8: Darknet marketplace dataset

I find the Dark net simply fascinating.

This is a data parse of marketplace data ripped from Agora (a dark/deep web) marketplace from the years 2014 to 2015. It contains drugs, weapons, books, services, and more.

Here's some inspiration:

• Description of this dataset: "This data set was made from an html rip made by reddit user "usheep" who threatened to expose all the vendors on Agora to the police if they did not meet his demands (sending him a small monetary amount ~few hundred dollars in exchange for him not leaking their info). Most information about what happened to "usheep" and his threats is nonexistent. He posted the html rip and was never heard from again. Agora shut down a few months after. It is unknown if this was related to "usheep" or not, but the raw html data remained." WOOAAAHH!
• Facebook hacking guide, ATM HACKING TUTORIAL, 50000 facebook likes, fake IDs, licences, lots of drugs and prostitution related entries -- the kinds of items in this dataset

9: News headlines of India that contains 18 years of headlines focussing on India

It contains approximately 2.9 million events published by Times of India from 2001 to 2018.

You could use this to -

• run a sentiment analysis over the headlines and see for yourself - Do the news agencies focus on bad news more than good news?
• understand what are the most popular topics in the Indian society
• chop this dataset into a smaller piece for a more focused analysis on categories like Bollywood, political parties, cricket and see the trend across the years

10: StackOverflow Developer Survey of more than 100,000 developers

You could use this meaty survey to arrive at data-backed answers to the following questions:

• Do people learn by contributing to open-source projects?
• How do opinions about AI differ across countries/age/dev roles?
• Views and opinions of the students (1 out every 5 responder in this survey is a student)
• How Vim users differ from non-Vim users?
• Create a salary predictor

I used it to make a comparison of software developers in India with the ones in US, UK, Germany and the entire world at large.

Source #3: Data is Plural

This is yet another source for Data Science or Machine Learning projects. It is a free email newsletter where the author sends you bunch of curious datasets each week.

Why you should analyse curious datasets for your personal projects:

1. They are thrilling to work on - you are curious about knowing the results of the analysis yourself
2. They are an easy way to create interesting projects - even a simple analysis on a dataset that is inherently interesting, will be interesting.

Alright, so here are some cool ones from Data is Plural's archives:

11: A dataset of 2,656 TED talks, with metadata and transcripts.

TED talks have become an integral part of our culture.

A group of teenagers cluster near their lockers, enjoying quick conversations between classes. One of them goes a little too long and, realizing it, addresses the group and the situation by announcing, "Well, thanks for coming to my TED talk." The rest laugh, nod their heads, and the conversational flow returns to normal before the bell sounds announcing that classes are about to begin. (From field notes by one of the authors.)

Analyse these transcripts to reveal some intricasies about our culture.

12: How couples meet and stay together. It is a survey of 4,002 adults, 3,009 of those had a spouse or main romantic partner. It even has follow-up surveys that were implemented one and two years after the main survey, to study couple dissolution rates.

An analysis can reveal answers to the following questions:

• Do traditional couples and nontraditional couples meet in the same way? What kinds of couples are more likely to have met online?
• Have the most recent marriage cohorts (especially the traditional heterosexual same-race married couples) met in the same way their parents and grandparents did?
• Does meeting online lead to greater or less couple stability?
• How do the couple dissolution rates of nontraditional couples compare to the couple dissolution rates of more traditional same-race heterosexual couples?
• How does the availability of civil union, domestic partnership or same-sex marriage rights affect couple stability for same-sex couples?

13: Electricity in rural India. The Smart Power India and the Initiative for Sustainable Energy Policy published a survey dataset that “covers 10,000 households and 2,000 rural enterprises across 200 villages in Bihar, Uttar Pradesh, Odisha, and Rajasthan.” Respondents were asked, among other things, how many hours per day they get electricity, whether they have solar panels, and the price they pay for kerosene.

Do an analysis to understand exactly how dire is the state of rural India and compare them with your own conditions.

14: Deaths on jobs Since 1992, the US Bureau of Labor Statistics’ has collected data on work-related deaths through its Census of Fatal Occupational Injuries.

You could do a detailed study of the jobs to avoid, maybe?

15: A dataset of sarcasms in TV shows like Friends and The Big Bang Theory! MUStARD is a corpus of 690 text and video clips “for research in automated sarcasm discovery.” The dataset’s 690 examples — half involving sarcasm, half not — come from Friends, The Golden Girls, The Big Bang Theory, and Sarcasmaholics Anonymous.

I bet there are lots of interesting things we could do with this hilarious dataset!

Source #4: Y. O. U.

Oh yes, I did that!

I wrote in the beginning - you can train your mind to come up with good ideas yourself.

I think Paul Graham's advice on how to find startup ideas also kinda applies to how to find your pet project ideas.

The way to get startup ideas is not to try to think of startup ideas. It's to look for problems, preferably problems you have yourself.

At the same time, (and this may sound like I'm contradicting myself) you don't want to set the bar too high. You may have watched the movie The Social Network too much and hope to make the next Google or Facebook out of this project. But you shouldn't. This will only slow down the learning, make you create unrealistic goals, and most dangerously, make you procrastinate.

Remember, your goal is not to write a billion dollar software. It is to create a program that is going to provide a stage for you to work on and simply learn from. Like, for instance:

16: While chatting with my friend, we discussed how cool it would be to build a tool to analyze our Whatsapp chats and reveal things like - #messages sent, #words sent, average no. of words per message, most common words, longest double texting streak, chat hour pattern, most shared website links and more. We later found out that we had rediscovered an idea that was really on popular on Reddit once

What's awesome was that in the process of building it, she tuned her  Python skills up a notch. Now, we might even try our hands on web development and build a website that allows anyone to run an analysis on their own chat file!

17: I use Chrome bookmarks a lot. I really need to add comments to my bookmarks so that I could save my motivation for bookmarking that awesome link. But Chrome browser doesn't have an option to comment. That is why I built a simple Chrome extension to help me add comments to my bookmarks!

And I tricked myself to learn- Javascript, jQuery, HTML

18: When Game of Thrones released its last season a few months ago, I thought of building a script to analyze the sentiment of tweets of various Game of Thrones seasons to learn just how bad the last season was (😜)
Do this and you can trick yourself to learn - Python, Machine Learning, NLP

19: And since your goal is to learn, you shouldn't feel bad about reimplementing some existing idea. One day I came across this popular post on Hacker News called "I taught my little brother JS and he built this videogame in a week". I checked out the game and it was kind of addictive but really simple. I told my above mentioned friend about it and we are building a Python version of this cool game using PyGame.

20: A simple app that reminds you to follow up with important, busy people that you want to connect with. I recently read an article by Alexey Guzey on how you shouldn't expect busy people to reply to your first message and how it is your responsibility to follow up with them. But when you have a bunch of important people to talk to, it can be a little difficult to keep track of the follow-ups. This app will do it for you and also remind you about future follow-ups.

3 pointers on how to come up with (sort of) cool ideas --

• Keep your eyes open
• Set a low bar
• Don't hesitate to reimplement

Do you have some problems like these yourself? It would be awesome if you tell the community about your weird, potentially cool project ideas in the comments below. I'll start with 2 of my own weird ideas. Check them out in the comments below! And let's make the comments of this post another goldmine of fantastic project ideas, huh?!

---

This is definitely not an exhaustive list of sources of cool project ideas. There are a lot more goldmines like this out there but, of course, they are difficult to find. I'll update this post as I discover more of them.

I'll announce any updates to this article on my Twitter, on Build To Learn newsletter and in Build To Learn Slack group.

Next in this series, I take apart the above projects one by one and give you a detailed roadmap of building and learning on the way. I start with the Whatsapp Chat Analyser project.

So, follow and subscribe to keep in touch.

Also you can reach out me on both Twitter and LinkedIn.