DEV Community

loading...

Data Scientists|Engineers: What are the Frameworks you use the most at your job?

strikingloo profile image Luciano Strika ・1 min read

I've seen a lot of statistics about programmers, but not specifically about Data Scientists or Engineers.
Because of that, I'd like to propose this survey:

  1. Do you identify as a Data Scientist or Data Engineer?
  2. What are the languages and frameworks you use the most at your job? Say, the top 5 or 6.

As a bonus:

  1. What are the Frameworks you wish you were using instead?

I will write back with some visuals or analysis if this gets enough traction.
Of course all the data will be public here, so you can do your own too.

Discussion

pic
Editor guide
Collapse
claudiodavi profile image
Claudio Davi

As a Machine Learning Engineer (I'm responsible for creating and deploying models) I use mostly Python with Flask and Tensorflow on Google Cloud Platform. This workflow has had some success in the past few projects that I've participated.

Collapse
strikingloo profile image
Luciano Strika Author

Do you feel GCP is more seamlessly integrated to TensorFlow than PyTorch? Or is it mostly agnostic? (I'm asking from a pretty ignorant standpoint)

Collapse
claudiodavi profile image
Claudio Davi

I'm not really experienced with PyTorch in production, but on my experience, Tensorflow feels natural on GCP. They have all kinds of services and support for it. For what I have seen, they have started to work on a framework agnostic machine learning hub. I've not experimented with PyTorch yet, but feels like I'll be doing it in a couple of months.

Collapse
mccurcio profile image
Matt Curcio

I am finishing by Masters in Bioinformatics and my department basically relies on R/Rstudio and Python and Docker. Quite a few people in the field also use Bash scripting quite a bit too, with a very little Perl too.

Collapse
strikingloo profile image
Luciano Strika Author

Awesome! It's interesting to see the perspective from someone who doesn't come from a computers/software background.
I also didn't think Bash was that widespread.

Collapse
fruiza profile image
Francisco Ruiz A
  1. A bit of both, although coming from a software eng background I tend to do more data eng than science.
  2. Python, Spark with PySpark, Databricks
Collapse
strikingloo profile image
Luciano Strika Author

Cool, that's a very data engineering like profile. What's Databricks used for?