What is Posenet?
https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
https://github.com/tensorflow/tfjs-models/tree/master/posenet
As you can see, basically we don't need to use nodejs for PoseNet since we need to use canvas to feed data to tensorflowjs lol
Also, we can use tensorflow with python instead of js, but I did try because there are nodejs and tfjs-node(https://github.com/tensorflow/tfjs-node).
steps
- install packages/libs
- install npm packages
- write code & run it
Step1
As I mentioned, need to use canvas, so need to install libs.
By the way, this is for mac, but you can find out information for Linux and Windows on the internet. In addition, now we can use Homebrew on Linux(I tested it on Ubuntu)
$ brew install pkg-config cairo pango libpng jpeg giflib
Step2
I'm using yarn since npm didn't allow me to install tfjs. But, you can use/try npm instead of yarn.
$ yarn add @tensorflow-models/posenet @tensorflow/tfjs @tensorflow/tfjs-node botkit canvas rollup
Step3
The code is messy since just test tfjs-node with posenet.
const tf = require('@tensorflow/tfjs-node');
const posenet = require('@tensorflow-models/posenet');
const {
createCanvas, Image
} = require('canvas')
const imageScaleFactor = 0.5;
const outputStride = 16;
const flipHorizontal = false;
const tryModel = async() => {
console.log('start');
const net = await posenet.load(0.75);
const img = new Image();
img.src = './test.jpg';
const canvas = createCanvas(img.width, img.height);
const ctx = canvas.getContext('2d');
ctx.drawImage(img, 0, 0);
const input = tf.browser.fromPixels(canvas);
const pose = await net.estimateSinglePose(input, imageScaleFactor, flipHorizontal, outputStride);
// console.log(pose);
for(const keypoint of pose.keypoints) {
console.log(`${keypoint.part}: (${keypoint.position.x},${keypoint.position.y})`);
}
console.log('end');
}
tryModel();
Result
Got 17 keypoints, yay!
Top comments (14)
i just used npm install @tensorflow-models/posenet to use the model.
nodejs v10.15.0
npm v6.9.0
I am always receiving this error: UnhandledPromiseRejectionWarning: Error: Image given has not completed loading
with my sample code?
Yes, after installing all the dependencies I ended up with this error code : FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal.
can you tell me your os version, nodejs version and your package.json's dependencies
linux subsystem on windows 10, "dependencies": {
"@tensorflow-models/posenet": "2.1.3",
"body-parser": "1.19.0",
"express": "4.17.1",
"morgan": "1.9.1",
"typescript": "3.6.3"
},
"devDependencies": {
"@tensorflow/tfjs": "1.2.9",
"@tensorflow/tfjs-node": "1.2.9",
"botkit": "4.5.0",
"canvas": "2.6.0",
"rollup": "1.21.2"
}
tensorflor-models/posenet has been updated so you need to update code.
just checked my
package.jsonI ended up uninstalling everything and reinstalling. But now when I run the code I get " (node:87485) UnhandledPromiseRejectionWarning: TypeError: Cannot create property 'architecture' on number '0.75'" which seems to be an error within posenet. maybe not everything is completely installed correctly? what do you think thank you for the help
It's because posenet api has been updated in version 2.0. See:
github.com/tensorflow/tfjs-models/...
Example new loading code:
Can you please add video inference support for posenet?
sorry i don't have time now.
i recommend you to check tfjs repo and dan's repos dev.to/oveddan since he is the peron who has implemented posenet to tfjs.
my code is outdated.
@gaelguedia
can you tell me your os version, nodejs version, npm/yarn and your package.json's dependencies
The above error was on windows 10, js 10. The issue seems to be related to canvas dependency. It's not easy to have it running on windows. It's easier on ubuntu especially easier to install cairo dependencies for Jpg/jpeg support