I want to finish reading the HTML standard. I'm working on my own browser optimized for web scraping, so knowing a lot about how a browser works is important.
By the way, the HTML standard is an excellent read. About a thousand pages of clear, detailed specifications of every single detail of what a browser does. Incredibly interesting and informative.
Fascinating. The HTML standard does look really interesting as I take a glance. I've been reading the CommonMark markdown spec myself lately. How did you first get interested in the browser project?
My company does a lot of web scraping, it's basically the entire business. Originally we were using Selenium and PhantomJS, but we started running into scaling issues. So now a scraping grid consists of 32 servers each with 8 cores and each costing hundreds of dollars a month. The servers are mostly at like 30% CPU usage. We have like 300k in free servers from various hosting companies so improving efficiency isn't too high priority, but something will have to be done eventually.
The obvious alternative to Selenium is to just make HTTP requests, but we have to crawl a lot of really crappy sites that use JavaScript for no apparent reason, and we want to be able to add a new site without spending a lot of time figuring out how to form spoof. So we're just making our own browser. It uses V8 to run JavaScript, which I had to write a Python C++ extension to do.
Admittedly it's not the most useful thing I could be doing. But it's hella fun.
1) Graduate (about to start my final semester)
2) Get a MVP version of my cooking helper app on at least 3 friends phones. V1 is for Android, and the server is written in Scala, both new for me. I recently got a copy of The Lean Startup, and hope to apply some of those ideas to the first release.
I'm interested in learning some functional programming, and felt this was a better option for a microservice than haskell. That's also on my to-learn list.
I'm intending to build a native app. Hoping to keep it simple, and make calls to a Scala API. I'm also hoping to make it so both the app & a web interface can use the same endpoints. I also don't have any experience, so this project is certainly experiment-oriented rather than launch.
My stated goals were to blog more, teach more, and finish a personal project. More details:
Blog more: I'm a firm believer in contributing to the C# and .NET communities, especially with how I work with both in my day job. How we utilize libraries and solve hard problems is something I find useful in sharing.
Teach more: I am a senior resource at my company. It's my goal to get my developer team members to get at or above my technical acumen and delivery skills.
Finish a personal project: I'm great at starting personal projects, typically ASP.NET websites. Time to finish one, hopefully before spring!
My personal realization that finally let me get finishing was to realize that projects I'd abandoned years ago, regardless of how misguided they seemed at the time, would have been awesome had I only kept grinding away at them. Eventually it would tip and get to cruise control. But by ditching them and starting over again, I was always starting over.
So I chose a project I knew would eventually turn out awesome if I just kept at it. That's this site, and it's still a work in progress, but it's getting a bit better every week.
Good observation on the finish vs restart. The other hard part about side projects is they can run over such a long timeframe that you can get demotivated by the difference in quality across components (code or otherwise). I think this can spur the temptation to start over, eg, "I'm so much better now! If I start from scratch it will be nothing but puppies and perfection!".
Catch being of course, you will continue to grow, especially if you are building things as a calling. Think the balance is to remember that if you're applying your trade you'll always be better than yesterday, so rework the critical bits (like gaping security or structural issues), and push forward to done.
Having a goal to open source something can compound this, since you know one day some bright young things are going to be lobbing entirely legitimate pull requests at your old, duct tape code from when you were first getting started.
My biggest goal for 2017 is to be more engaged in the community. I hope to do so by blogging experiences, speaking at conferences and meetups, and more contributions to open source.
This past year I have dipped my toe on some of these but I want to submerge myself and commit to it.
Open source is something I attempted before but I consistently felt I was not good enough to take on any of the issues and didn't dedicate enough time to it. After attending my first conference (since HS), I found my place or to put it better I found my passion. I took my first steps toward speaking this year on a FP panel at a local conference and also gave a talk on RRv4 at a local meet up.
I think mentoring/teaching has always been a strong point for me but it was typically focused to within the workspace. My self-awareness has gotten me past any of the imposter syndrome I had, which has me ready to share with a larger audience.
Nicolai is a thirty year old boy, as the narrator would put it, who has found his passion in software development. He constantly reads, thinks, and writes about it, and codes for a living as well a...
I've started a few side projects and want to focus on making them usable. This is where they stand as of now:
WorkflowyFX should one day become a Chrome / Firefox plugin that makes using the awesome Workflowy even awesomer. At the moment it does almost nothing and must be installed locally. Mpf.
JUnit Io is where I want to collect JUnit 5 extensions. The project is almost set up, including publishing nightly snapshots, and ready to gather features. But as with WorkflowyFX, nothing usable was published yet.
I don't even know whether the third thingy is worth mentioning. It's hardly a project and more of a learning exercise. I thought it would be nice to have a tool that lists recent releases of the biggest open source projects so I wrote something in Kotlin (my first time) that asynchronously (my first time) connects to GitHub (you can guess). Not least thanks to a friend I made good progress just yesterday and it may become usable soon. Ish.
With WorkflowyFX, I have to get it into two browser stores (Chrome, Firefox), which I'm sure will be an "interesting" experience. JUnit Io needs some more infrastructure (documentation, website, ...) but from then on it should be "just coding" as publication is automated. Finally, the "recent releases" thing just needs to be finished and put onto GitHub.
I would like to continue to dabble in Kotlin and JavaScript (I'm a one-trick Java pony) and get to at least a decent level of productivity in both languages.
create more feminine @NERDpraunig developer t-shirts: nerdpraunig.com
In general I want to focus more on my health - I am currently reading "The Healthy Programmer.
And I want to blog again and share my very positive developer experience, like my story "From Secretary to Software Developer: the hard way" which I submitted to medium: medium.com/code-like-a-girl/from-s...
Good luck with the app release process, Apple makes you jump though hoops 😒
I love that if/else tee!
Would you recommend that book so far? I'm always on the lookout for programming-related books that come in audiobook form and that one does.
Congrats on your journey so far, and feel free to use this platform to tell more of your story, as well as blog about Swift and share your burgeoning expertise with the language.
In 2017 my goal is to contribute to open source projects. As an avid user of R, I reap the benefits of open source materials on the daily. And, lo and behold, just by asking the rstats twitterverse for ideas, I quickly got a bevy of great suggestions-- all of which were less intimidating than my internal vision of what "contributing" to open source might entail. Issue fixes, building extensions, refactoring old code etc. are all well and good, but starting out by lending a hand with documentation, or reviewing packages makes this goal seem much more approachable.
No project ideas, but what about small daily challenges instead? E.g., Project Euler, Advent of Code 2015 or 2016 (always open). You'd get your daily dose of new problem solving. You could event implement solutions to some of the puzzles in different languages and create a little cookbook of neat-o things.
Get better at time management. I'm often underestimating the length of a time a project will take because of changes in scope. (I get paid for the out of scope work, but it still pushes the launch date back.) It's resulting in a significant amount of project overlap.
Build my first public Shopify app. I've built two apps for clients, but I'd like to sell my first app in 2017. I have the idea, I just need to figure out how to build it (using Ruby on Rails).
Get more involved in the local dev community. Atlanta has a strong community of developers and I'm spending too much time on the sidelines. I'd love to join in on a hackathon or help out with an after school coding program.
Oldest comments (102)
I want to finish reading the HTML standard. I'm working on my own browser optimized for web scraping, so knowing a lot about how a browser works is important.
By the way, the HTML standard is an excellent read. About a thousand pages of clear, detailed specifications of every single detail of what a browser does. Incredibly interesting and informative.
Fascinating. The HTML standard does look really interesting as I take a glance. I've been reading the CommonMark markdown spec myself lately. How did you first get interested in the browser project?
My company does a lot of web scraping, it's basically the entire business. Originally we were using Selenium and PhantomJS, but we started running into scaling issues. So now a scraping grid consists of 32 servers each with 8 cores and each costing hundreds of dollars a month. The servers are mostly at like 30% CPU usage. We have like 300k in free servers from various hosting companies so improving efficiency isn't too high priority, but something will have to be done eventually.
The obvious alternative to Selenium is to just make HTTP requests, but we have to crawl a lot of really crappy sites that use JavaScript for no apparent reason, and we want to be able to add a new site without spending a lot of time figuring out how to form spoof. So we're just making our own browser. It uses V8 to run JavaScript, which I had to write a Python C++ extension to do.
Admittedly it's not the most useful thing I could be doing. But it's hella fun.
Well whether or not this specific activity is "useful", I'm sure you'll get a hell of a lot out of reading the whole HTML standard!
To clarify, the standard found here? html.spec.whatwg.org/multipage/
That's good to know it's a solid & clear read, I'll have to make some time to work through it.
That's the one.
1) Graduate (about to start my final semester)
2) Get a MVP version of my cooking helper app on at least 3 friends phones. V1 is for Android, and the server is written in Scala, both new for me. I recently got a copy of The Lean Startup, and hope to apply some of those ideas to the first release.
Lean Startup is definitely critical reading for this kind of thing. Good luck with the launch. What made you choose Scala for the server?
I'm interested in learning some functional programming, and felt this was a better option for a microservice than haskell. That's also on my to-learn list.
Clojure is an interesting and productive lisp. You might find the principles there intriguing as well.
Aah, that's great to know! I'll look into that some more - I've had coworkers apply some Clojure styles & ideas to our codebase with great results.
Is it a native app or u use some Scala framework?
I do not have any experience with apps, I would like to know more :)
I'm intending to build a native app. Hoping to keep it simple, and make calls to a Scala API. I'm also hoping to make it so both the app & a web interface can use the same endpoints. I also don't have any experience, so this project is certainly experiment-oriented rather than launch.
ok, good luck with that! The common REST/graphql API endpoint is a good idea.
My stated goals were to blog more, teach more, and finish a personal project. More details:
Blog more: I'm a firm believer in contributing to the C# and .NET communities, especially with how I work with both in my day job. How we utilize libraries and solve hard problems is something I find useful in sharing.
Teach more: I am a senior resource at my company. It's my goal to get my developer team members to get at or above my technical acumen and delivery skills.
Finish a personal project: I'm great at starting personal projects, typically ASP.NET websites. Time to finish one, hopefully before spring!
My personal realization that finally let me get finishing was to realize that projects I'd abandoned years ago, regardless of how misguided they seemed at the time, would have been awesome had I only kept grinding away at them. Eventually it would tip and get to cruise control. But by ditching them and starting over again, I was always starting over.
So I chose a project I knew would eventually turn out awesome if I just kept at it. That's this site, and it's still a work in progress, but it's getting a bit better every week.
Good observation on the finish vs restart. The other hard part about side projects is they can run over such a long timeframe that you can get demotivated by the difference in quality across components (code or otherwise). I think this can spur the temptation to start over, eg, "I'm so much better now! If I start from scratch it will be nothing but puppies and perfection!".
Catch being of course, you will continue to grow, especially if you are building things as a calling. Think the balance is to remember that if you're applying your trade you'll always be better than yesterday, so rework the critical bits (like gaping security or structural issues), and push forward to done.
Having a goal to open source something can compound this, since you know one day some bright young things are going to be lobbing entirely legitimate pull requests at your old, duct tape code from when you were first getting started.
My biggest goal for 2017 is to be more engaged in the community. I hope to do so by blogging experiences, speaking at conferences and meetups, and more contributions to open source.
This past year I have dipped my toe on some of these but I want to submerge myself and commit to it.
And I think you're doing a great job with it, and as you know, Jess and I are here to help you make the most of our channels.
May I ask why you decided to make this a focus? Have you tried to deliberately do much of this stuff in the past?
Open source is something I attempted before but I consistently felt I was not good enough to take on any of the issues and didn't dedicate enough time to it. After attending my first conference (since HS), I found my place or to put it better I found my passion. I took my first steps toward speaking this year on a FP panel at a local conference and also gave a talk on RRv4 at a local meet up.
I think mentoring/teaching has always been a strong point for me but it was typically focused to within the workspace. My self-awareness has gotten me past any of the imposter syndrome I had, which has me ready to share with a larger audience.
Ooooh those are really great goals. How did you wind up with the goal of teaching your friends, did they approach you?
Elixir/Elm seems like the hot web stack for 2017. What got you interested in those languages in particular?
I have a friend which actually do elixir and she's learning React... but she does not know anything about js ecosystem. So I want to help her.
And about erlang/elixir/elm: currently i am working as front end dev and we have the backend writen in elixir. Its a great opportunity.
I've started a few side projects and want to focus on making them usable. This is where they stand as of now:
With WorkflowyFX, I have to get it into two browser stores (Chrome, Firefox), which I'm sure will be an "interesting" experience. JUnit Io needs some more infrastructure (documentation, website, ...) but from then on it should be "just coding" as publication is automated. Finally, the "recent releases" thing just needs to be finished and put onto GitHub.
I would like to continue to dabble in Kotlin and JavaScript (I'm a one-trick Java pony) and get to at least a decent level of productivity in both languages.
I've never distributed a Firefox extension, but the Chrome store was a breeze. Quick and simple process.
My goals for 2017 are:
In general I want to focus more on my health - I am currently reading "The Healthy Programmer.
And I want to blog again and share my very positive developer experience, like my story "From Secretary to Software Developer: the hard way" which I submitted to medium: medium.com/code-like-a-girl/from-s...
Good luck with the app release process, Apple makes you jump though hoops 😒
I love that if/else tee!
Would you recommend that book so far? I'm always on the lookout for programming-related books that come in audiobook form and that one does.
Congrats on your journey so far, and feel free to use this platform to tell more of your story, as well as blog about Swift and share your burgeoning expertise with the language.
2018 checking in. How did 2017 go?
I have a few small attainable goals for 2017
What are the personal projects?
In 2017 my goal is to contribute to open source projects. As an avid user of R, I reap the benefits of open source materials on the daily. And, lo and behold, just by asking the rstats twitterverse for ideas, I quickly got a bevy of great suggestions-- all of which were less intimidating than my internal vision of what "contributing" to open source might entail. Issue fixes, building extensions, refactoring old code etc. are all well and good, but starting out by lending a hand with documentation, or reviewing packages makes this goal seem much more approachable.
My first contribution to a "big" open source project was adding some events to the calendar on the Reactjs website. 😊
Yesterday I wrote a post about my 2017 developer goals :D mydevelopment.blog/my-developer-go...
Start:
Stop:
Continue:
Get sodium_compat finished, make WordPress's automatic update secure against infrastructure attacks.
1) Release an app to the Appstore. (problem is, haven't got any idea of an app)
2) Learn how to write cleaner and more readable code.
None - I have literally no ideas for projects D:
No project ideas, but what about small daily challenges instead? E.g., Project Euler, Advent of Code 2015 or 2016 (always open). You'd get your daily dose of new problem solving. You could event implement solutions to some of the puzzles in different languages and create a little cookbook of neat-o things.
Somehow I can't even find those interesting. I am a weird person
Get better at time management. I'm often underestimating the length of a time a project will take because of changes in scope. (I get paid for the out of scope work, but it still pushes the launch date back.) It's resulting in a significant amount of project overlap.
Build my first public Shopify app. I've built two apps for clients, but I'd like to sell my first app in 2017. I have the idea, I just need to figure out how to build it (using Ruby on Rails).
Get more involved in the local dev community. Atlanta has a strong community of developers and I'm spending too much time on the sidelines. I'd love to join in on a hackathon or help out with an after school coding program.
Regarding time management...