Not only the error mentioned in title, a warning would pop up before it:

Warning-Potential Nesting Problem

Due to the Accuracy and Margin you have set being too high for this computer to handle you may experience unpredictable results.

Please reduce the accuracy or increase the margin of nesting in future.

Details:

Default: SUS304 1.5  Margin=0.00
Default: SUS201 1.0 Margin=0.00

TL;DR:

Click File > Setup > Installed Machines > (Select the machine) > Tools... > (Select the Tool) > Setup... > Globals

If "Nesting Margin" is visible in the Selected list, then remove it.

Now, the nesting margin is determined by the value you set each time you click "Automatic Nesting":

Additional Resource

If you wanna set difference margins for each material, refer to this doc from Autodesk