Hey I was not posting for a while, because I’ve started working on open sourcing my side project called Videolitic. I’ve rewritten it to Swift + Combine + SwiftUI from Rx with UIKit so there is a lot of room for improvement - I’ve never done anything in SwiftUI and Combine before.
Videolitic. is an on-device video analysis framework. I am tracking each person from the video and detect his or her age, emotions and ethnicity, I am also creating transcript of the video. All results are combined and time-based.
This is how it works!
My main goal is to create better open source models build on top of Vision which can detect:
- Age
- Gender
- Ethnicity
- Emotions
I wasn't able to OpenSource models which was trained on better datasets
The project is only library - I’ve added some UI just to present the results. There is a lot of Core ML + Vision + Create ML based on transfer learning.
I also integrated for the first time Travis-CI and it's GREAT!
I will create tutorial how to integrate Tavis-CI with the open source GitHub project and some articles with most interesting snippets!
Please checkout GitHub and contribute!
BTW! If you like my articles please follow me on Twitter Thanks!
Top comments (8)
This is a very interesting project and worth mentioning in a wider community.
My personal opinion regarding ethnicity classification is that it is nearly impossible to execute as many races have similar visual criteria but different origins and characteristics.
For example, Yemen people are confused by machine learning with Africans/African Americans etc.
Consider also the fact that within 2 or 3 decades most people in western economic countries will be a mix of races and looka cannot point to correct ethnicity classifications.
Gender is not yet an issue but this is also a changing subject.
I would extract ethnicity classification as this is impossible to predict.
Hey thank you for a kind word! I don't know how to mention it in a wider community but I will try to 👌
This is very interesting point of view, I always assumed that ethnicity is as simple as gender but probably as you wrote it's much harder to distinguish that with good confidence.
On the other hand maybe machine learning will be growing with us and more mix of races will be also visible in outputs of ML models - for example model has 60% certainty that someone is white and 40% that someone is asian because this person is a mix of races. As I wrote it's very interesting observation and may be no point in trying to create perfect model.
Thanks! :)
Wait, is Hillary Clinton black? 🤔
Yeah, for ethnicity I wasn’t able to use better model I’ve created to my non open source project because the dataset was created on different license:) Everything depends on datasets - I didn’t have time to create a script that can prepare a better one, but I was hoping maybe someone from the community has!
Can we talk about how it's race and age predictions are so far from correct? 😄
Age model usually does a better job, but maybe it’s because she is wrinkle-free 🙂 about the ethnicity same answer I wrote below - dataset was bad :)
Great project 👏
Thank you!!