My exploratory work tends to fall in one of three categories:
It's all goal directed against a mental model of what I'm doing. For (1), it's trying to eliminate swathes of the control flow and data flow of the system as quickly as possible. For (2), it's getting rough estimates of the economic impact of an idea. For (3), it's trying to find situations that will break a model, rather like the adversarial process a security researcher goes through.
Instead of writing up a huge list of things to explore, treat it the way Tukey did in his book Exploratory Data Analysis that started the field. You start with a data set, and sequentially ask questions about it and answer them as fast as possible, since often you can cut off a whole line of questioning at the very beginning with a quick check. (It's a great book, by the way...especially to see what he developed to do this kind of work by hand.)
I have seen references to John Tukey's book in my reading. It does look interesting.
Hi Fred, I am curious. What other Data Science blogs or sites do you follow?
I am curious. What other Data Science blogs or sites do you follow?
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.