DEV Community

loading...

Run PoseNet with Nodejs

kojikanao profile image Koji (he/him) ・2 min read

What is Posenet?

https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
https://github.com/tensorflow/tfjs-models/tree/master/posenet

As you can see, basically we don't need to use nodejs for PoseNet since we need to use canvas to feed data to tensorflowjs lol

Also, we can use tensorflow with python instead of js, but I did try because there are nodejs and tfjs-node(https://github.com/tensorflow/tfjs-node).

steps

  1. install packages/libs
  2. install npm packages
  3. write code & run it

Step1

As I mentioned, need to use canvas, so need to install libs.
By the way, this is for mac, but you can find out information for Linux and Windows on the internet. In addition, now we can use Homebrew on Linux(I tested it on Ubuntu)

$ brew install pkg-config cairo pango libpng jpeg giflib
Enter fullscreen mode Exit fullscreen mode

Step2

I'm using yarn since npm didn't allow me to install tfjs. But, you can use/try npm instead of yarn.

$ yarn add @tensorflow-models/posenet @tensorflow/tfjs @tensorflow/tfjs-node botkit canvas rollup
Enter fullscreen mode Exit fullscreen mode

Step3

The code is messy since just test tfjs-node with posenet.

const tf = require('@tensorflow/tfjs-node');
const posenet = require('@tensorflow-models/posenet');
const {
    createCanvas, Image
} = require('canvas')
const imageScaleFactor = 0.5;
const outputStride = 16;
const flipHorizontal = false;

const tryModel = async() => {
    console.log('start');
    const net = await posenet.load(0.75);
    const img = new Image();
    img.src = './test.jpg';
    const canvas = createCanvas(img.width, img.height);
    const ctx = canvas.getContext('2d');
    ctx.drawImage(img, 0, 0);
    const input = tf.browser.fromPixels(canvas);
    const pose = await net.estimateSinglePose(input, imageScaleFactor, flipHorizontal, outputStride);
    // console.log(pose);
    for(const keypoint of pose.keypoints) {
        console.log(`${keypoint.part}: (${keypoint.position.x},${keypoint.position.y})`);
    }
    console.log('end');
}



tryModel();
Enter fullscreen mode Exit fullscreen mode

Result

Got 17 keypoints, yay!

Discussion (14)

pic
Editor guide
Collapse
Sloan, the sloth mascot
Comment deleted
Collapse
kojikanao profile image
Koji (he/him) Author

i just used npm install @tensorflow-models/posenet to use the model.

nodejs v10.15.0
npm v6.9.0

Collapse
gaelguedia profile image
Geekster🇨🇲

I am always receiving this error: UnhandledPromiseRejectionWarning: Error: Image given has not completed loading

Collapse
kojikanao profile image
Koji (he/him) Author

with my sample code?

Collapse
gaelguedia profile image
Geekster🇨🇲

Yes, after installing all the dependencies I ended up with this error code : FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal.

Thread Thread
kojikanao profile image
Koji (he/him) Author

can you tell me your os version, nodejs version and your package.json's dependencies

Thread Thread
gaelguedia profile image
Geekster🇨🇲

linux subsystem on windows 10, "dependencies": {
"@tensorflow-models/posenet": "2.1.3",
"body-parser": "1.19.0",
"express": "4.17.1",
"morgan": "1.9.1",
"typescript": "3.6.3"
},
"devDependencies": {
"@tensorflow/tfjs": "1.2.9",
"@tensorflow/tfjs-node": "1.2.9",
"botkit": "4.5.0",
"canvas": "2.6.0",
"rollup": "1.21.2"
}

Thread Thread
kojikanao profile image
Koji (he/him) Author

tensorflor-models/posenet has been updated so you need to update code.

just checked my package.json

"dependencies": {
    "@tensorflow-models/posenet": "^1.0.3",
    "@tensorflow/tfjs": "^1.1.2",
    "@tensorflow/tfjs-node": "^1.1.2",
    "botkit": "^4.0.2",
    "canvas": "^2.5.0",
    "rollup": "^1.13.1"
  }
Collapse
yalomdd profile image
yalomdd

I ended up uninstalling everything and reinstalling. But now when I run the code I get " (node:87485) UnhandledPromiseRejectionWarning: TypeError: Cannot create property 'architecture' on number '0.75'" which seems to be an error within posenet. maybe not everything is completely installed correctly? what do you think thank you for the help

Collapse
oveddan profile image
Dan Oved

It's because posenet api has been updated in version 2.0. See:
github.com/tensorflow/tfjs-models/...

Example new loading code:

const net = await posenet.load({
  architecture: 'MobileNetV1',
  outputStride: 16,
  inputResolution: 513,
  multiplier: 0.75
});
Collapse
cvramanan profile image
Venkataramanan Chockalingam

Can you please add video inference support for posenet?

Collapse
kojikanao profile image
Koji (he/him) Author

sorry i don't have time now.
i recommend you to check tfjs repo and dan's repos dev.to/oveddan since he is the peron who has implemented posenet to tfjs.
my code is outdated.

Collapse
kojikanao profile image
Koji (he/him) Author

@gaelguedia
can you tell me your os version, nodejs version, npm/yarn and your package.json's dependencies

Collapse
gaelguedia profile image
Geekster🇨🇲

The above error was on windows 10, js 10. The issue seems to be related to canvas dependency. It's not easy to have it running on windows. It's easier on ubuntu especially easier to install cairo dependencies for Jpg/jpeg support