FluidNC: Troubleshooting $MI / $MD commanding weirdnesses

Software / Firmware Jackpot Board / FluidNC
21

I suppose I can try this again with the MPCNC config on my test board. If I can’t get to it tonight, I’ll do the test tomorrow.

I’m going to put an order in this evening and will be picking up some V1 ESP32s. I’m planning to break out the CP210x UART GPIO and might play with making a test harness so that I can measure all of the enables under automated control. I’ll probably also modify a set of TMC2209s with long pins on the headers so that I can land logic analyzer and those GPIOs I mention above.

22

Well… more bugs.

I loaded the MPCNC config on my test jackpot.

Everything inits OK with this config on boot.
If I then send $MD, all 5 drivers EN goes to 3.3V, and things are disabled.
I then send $MI, and all 5 drivers enable and the EN goes to 0.0V
I send a $MD, and the Z position EN pin on the jackpot stays at 3.3V while the other four go to 0.0V!

This is repeatable, I’ve now done this 15 boots in a row, and had this result each and every time.

23

Here is the $ss for when the z and y axes don’t move 10mm:

[MSG:INFO: FluidNC 3.7.14-webui3-test https://github.com/bdring/FluidNC]]
[MSG:INFO: Compiled with ESP32 SDK:v4.4.4]]
[MSG:INFO: Local filesystem type is littlefs]]
[MSG:INFO: Configuration file:config.yaml]]
[MSG:INFO: Machine MPCNC]]
[MSG:INFO: Board Jackpot TMC2209]]
[MSG:INFO: UART1 Tx:gpio.0 Rx:gpio.4 RTS:NO_PIN Baud:115200]]
[MSG:INFO: I2SO BCK:gpio.22 WS:gpio.17 DATA:gpio.21]]
[MSG:INFO: SPI SCK:gpio.18 MOSI:gpio.23 MISO:gpio.19]]
[MSG:INFO: SD Card cs_pin:gpio.5 detect:NO_PIN freq:20000000]]
[MSG:INFO: Stepping:I2S_static Pulse:4us Dsbl Delay:0us Dir Delay:1us Idle Delay:255ms]]
[MSG:INFO: Axis count 4]]
[MSG:INFO: Axis X (0.000,490.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:0 CS:NO_PIN Step:I2SO.2 Dir:I2SO.1 Disable:I2SO.0 R:0.110]]
[MSG:INFO:  X Neg Limit gpio.25]]
[MSG:INFO:   Motor1]]
[MSG:INFO:     tmc_2209 UART1 Addr:3 CS:I2SO.22 Step:I2SO.21 Dir:I2SO.20 Disable:I2SO.23 R:0.110]]
[MSG:INFO:  X2 Neg Limit gpio.35]]
[MSG:INFO: Axis Y (0.000,490.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:1 CS:NO_PIN Step:I2SO.5 Dir:I2SO.4 Disable:I2SO.7 R:0.110]]
[MSG:INFO:  Y Neg Limit gpio.33]]
[MSG:INFO:   Motor1]]
[MSG:INFO:     tmc_2209 UART1 Addr:3 CS:I2SO.19 Step:I2SO.18 Dir:I2SO.17 Disable:I2SO.16 R:0.110]]
[MSG:INFO:  Y2 Neg Limit gpio.34]]
[MSG:INFO: Axis Z (0.000,200.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:2 CS:NO_PIN Step:I2SO.10 Dir:I2SO.9 Disable:I2SO.8 R:0.110]]
[MSG:INFO:  Z Neg Limit gpio.32:low]]
[MSG:INFO: Axis A (-720.000,0.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:3 CS:I2SO.14 Step:I2SO.13 Dir:I2SO.12 Disable:I2SO.15 R:0.110]]
[MSG:ERR: X Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: X2 Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: Y Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: Y2 Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: Z Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: A Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:INFO: Kinematic system: Cartesian]]
[MSG:INFO: Laser Ena:NO_PIN Out:gpio.27 Freq:5000Hz Period:8191]]
[MSG:INFO: Using spindle Laser]]
[MSG:INFO: Probe Pin: gpio.36:low]]
[MSG:INFO: Connecting to STA SSID:ATTUvwG8BA]]
[MSG:INFO: Connecting.]]
[MSG:INFO: Connecting..]]
[MSG:INFO: Connected - IP is 192.168.1.177]]
[MSG:INFO: WiFi on]]
[MSG:INFO: HTTP started on port 80]]
[MSG:INFO: Telnet started on port 23]]```

Here it is after $MD and $MI:
```$ss
[MSG:INFO: FluidNC 3.7.14-webui3-test https://github.com/bdring/FluidNC]]
[MSG:INFO: Compiled with ESP32 SDK:v4.4.4]]
FO: Local filesystem type is littlefs]]
[MSG:INFO: Configuration file:config.yaml]]
[MSG:INFO: Machine MPCNC]]
[MSG:INFO: Board Jackpot TMC2209]]
[MSG:INFO: UART1 Tx:gpio.0 Rx:gpio.4 RTS:NO_PIN Baud:115200]]
[MSG:INFO: I2SO BCK:gpio.22 WS:gpio.17 DATA:gpio.21]]
[MSG:INFO: SPI SCK:gpio.18 MOSI:gpio.23 MISO:gpio.19]]
[MSG:INFO: SD Card cs_pin:gpio.5 detect:NO_PIN freq:20000000]]
[MSG:INFO: Stepping:I2S_static Pulse:4us Dsbl Delay:0us Dir Delay:1us Idle Delay:255ms]]
[MSG:INFO: Axis count 4]]
[MSG:INFO: Axis X (0.000,490.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:0 CS:NO_PIN Step:I2SO.2 Dir:I2SO.1 Disable:I2SO.0 R:0.110]]
[MSG:INFO:  X Neg Limit gpio.25]]
[MSG:INFO:   Motor1]]
[MSG:INFO:     tmc_2209 UART1 Addr:3 CS:I2SO.22 Step:I2SO.21 Dir:I2SO.20 Disable:I2SO.23 R:0.110]]
[MSG:INFO:  X2 Neg Limit gpio.35]]
[MSG:INFO: Axis Y (0.000,490.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:1 CS:NO_PIN Step:I2SO.5 Dir:I2SO.4 Disable:I2SO.7 R:0.110]]
[MSG:INFO:  Y Neg Limit gpio.33]]
[MSG:INFO:   Motor1]]
[MSG:INFO:     tmc_2209 UART1 Addr:3 CS:I2SO.19 Step:I2SO.18 Dir:I2SO.17 Disable:I2SO.16 R:0.110]]
[MSG:INFO:  Y2 Neg Limit gpio.34]]
[MSG:INFO: Axis Z (0.000,200.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:2 CS:NO_PIN Step:I2SO.10 Dir:I2SO.9 Disable:I2SO.8 R:0.110]]
[MSG:INFO:  Z Neg Limit gpio.32:low]]
[MSG:INFO: Axis A (-720.000,0.000)]]
[MSG:INFO:   Motor0]]
[MSG:INFO:     tmc_2209 UART1 Addr:3 CS:I2SO.14 Step:I2SO.13 Dir:I2SO.12 Disable:I2SO.15 R:0.110]]
[MSG:ERR: X Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: X2 Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: Y Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: Y2 Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: Z Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:ERR: A Axis TMC driver not detected - expected 0x21 got 0x0]]
[MSG:INFO: Kinematic system: Cartesian]]
[MSG:INFO: Laser Ena:NO_PIN Out:gpio.27 Freq:5000Hz Period:8191]]
[MSG:INFO: Using spindle Laser]]
[MSG:INFO: Probe Pin: gpio.36:low]]
[MSG:INFO: Connecting to STA SSID:ATTUvwG8BA]]
[MSG:INFO: Connecting.]]
[MSG:INFO: Connecting..]]
[MSG:INFO: Connected - IP is 192.168.1.177]]
[MSG:INFO: WiFi on]]
[MSG:INFO: HTTP started on port 80]]
[MSG:INFO: Telnet started on port 23]]```

Both state TMC driver not detected. Weird!

But I get this message after I issue the $MI command:

[MSG:INFO: X Axis driver test passed]
[MSG:INFO: X2 Axis driver test passed]
[MSG:INFO: Y Axis driver test passed]
[MSG:INFO: Y2 Axis driver test passed]
[MSG:INFO: Z Axis driver test passed]
[MSG:INFO: A Axis driver test passed]
24

Yes, this is quite weird. You have a board where the TMC driver UART communication is not working at least some of the time.

This means the TMCs should run with their default stored configuation, or if board power has stayed on they will stick with the last commanded settings.

Have you ever replaced/swapped a TMC driver on this board? It acts like you have one TMC driver with different stored settings than the others.

When this driver init failure happens, it seems FluidNC is still toggling the ste/enable/dir bits even though the driver init failed. That doesn’t seem right! More and more curious as we go.

25

I have 6 TMC drivers installed. The A axis is my rotary.

It only does the ERR: TMC driver not detected if I plug in the USB first. If I power it up and then plug in the USB it doesn’t show that error. I can run $MD and $MI without any errors.

I just retried $MD and the motors weren’t disabled. IDK!!!

26

OK, this part may be expected. There’s a diode on the Jackpot that prevents the board from being back-powered by the ESP32 USB as the USB port can’t source enough current to run the electronics.

If you power up the Jackpot board after connecting USB, but don’t do a $MI, then the TMC drivers will run with whatever their stored settings are. So your system not moving correctly at this point is actually somewhat expected. The perhaps mildly unexpected part is that FluidNC still sends EN/STEP/DIR for stepper drivers that are defined but which failed to initialize. I’m not sure this is intended behavior.

The question is, if you connect the USB, then apply Jackpot board power, then send $MI, then I think FluidNC is expected to be able to initialize the steppers. If you follow this sequence and then try motion, does it move correctly for you?

I’m realizing that $SS is only reporting the initial boot and doesn’t reflect the changes made along the way.

27

Yes it does.

2 Likes
28

I did some more testing on this yesterday, after I received a shipment from V1.

I now have two test jackpot boards, and older blue one and a shiny new black one. I also have a collection of ESP32s. This includes USB-C, a no-name clone, the V1E white test ESP32, and a genuine ESP-32 dev board.

My test setup had the blue and black Jackpots, and all of the ESP32s described above. I tested 3.7.8, 3.7.12, 3.7.13 and the webuitest version of 3.7.14.

ALL of these various configurations are repeatable for me with the bug we’ve seen, which to summarize looks like the following: (LR3 config.yaml)

  • Power on a board from the external power supply. Board initializes fine. All stepper drivers enable.
  • Connect FluidTerm so I can watch all of this…
  • Send a $MD- all of the steppers disable as expected.
  • Send a $MI- all of the motors enable as expected.
  • Send a $MD- none of the EN pins on the TMC2209s toggle, they stay at 0.0V
  • Send a $MI- FluidNC reports the drivers have all initialized correctly, but I can’t notice any change- driver ENs were already at 0.0V
  • Send a Control-R (Restarts the MCU)- system resets, and FluidNC comes up happy.
  • Send a $MD- FluidNC reports the drivers are disabled, and I see the TMC2209s EN toggle to 3.3V
  • Repeat the rest of the sequence as above, and the behavior repeats.

I don’t see how this can be anything but a bug in FluidNC.

2 Likes