Jackpot 3 changes?

It’ll definitely help, but I have no idea how much. That’s a good point, that’s probably what I’d do, anyway.

What are the probes you’re using? Sadly, this is a case of getting what you pay for and the cheap ones are really bad.

I have a couple of sets of this style but supplied with decent test equipment (keysight, saleae, etc.) which work well:


https://www.digikey.co.nz/en/products/detail/e-z-hook/XKM-S/528233

Those are super easy to use on SOIC style packages and work ok on TSSOPs like the driver ICs with a bit of care. Attach them with the board powered down and use some kapton tape to avoid them swiveling and coming off or contacting the adjacent pins.

The cheap Aliexpress versions of those are uniformly horrible in my experience, though. I keep wanting them to be good, I can’t see a reason for them to be that expensive, but clearly there’s something there.

Another option I have is the Sensepeek PCBite probing setup:


https://sensepeek.com/pcbite-kit-with-4x-sq10-probes-and-test-wires
I have the version with a bunch of logic probes, a couple of multimeter probes and 4 scope probes. It’s a bit of a pain in the ass to set up but it works pretty well, all told.

The final option: Put test points on your board. I’m sure I’ve made that point a ton of times, as well :slight_smile: I know this isn’t particularly conducive towards making the smallest board possible, but I usually put either through-hole or SMD test points for internal traces where I know I’ll likely need to get to them for dev, debug or troubleshooting. You don’t need to have the test points assembled, you can just add them as needed, providing the space is there. My preferred ones are the Keystone ones:
Through-hole:

Miniature for regular signals
Compact for voltage rails
Multipurpose for 0V/return rails, usually 3-4 around the board
SMD:

I pretty much exclusively use the micro-miniature with the pad shrunk to match the test point size… With the shrunken pad these need to be very carefully used or they’ll just pull off the board, but they work well enough.

Alternatively, just adding a ~1mm circular SMD pad is a good approach. This is how we do connections for test fixtures using pogo pins with the idea that each pad will have a pogo pin jabbing into it. They work great with the Sensepeek setup from above or manually probing by hand, too, but obviously anything more than 1 signal and you’re in a bit of a pickle.

Another option I’ve never used, personally, would be something like a test clip. No idea if those exist for TSSOPs.

Some other notes:
You don’t have to probe directly at the chip, perhaps find somewhere else like a via that you can probe. It’s best to probe at the device in question but with care you can probe elsewhere.
It’s a good idea if you’re probing elsewhere to first manually verify the integrity of your traces. If you’re probing at the source of the signal or partway along a trace, make sure it looks the same at the device. Over-etched traces or traces with hairline shorts do occur sometimes, especially with cheaper boards. Broken via barrels are common and can be notoriously to diagnose because they can be intermittent, temperature related and the pressure from probing can change their behaviour.
30AWG Wire-wrap wire is good for this kind of thing because it’s stiff solid core wire that strips easily and you can tape it down, tack it onto a pin with solder and then you’ve got an external probing point. Best avoided with anything too high of a frequency like up into the 10s/100s of MHz, though.

With troubleshooting, it’s best to kinda look at everything. The goal is to try not to get blinkered into thinking ‘it must be this’, or ‘this is right so it must be a dead chip’ etc.

I like the approach of having a bunch of options to try, even writing them down if I have to, and ordering them from most likely to least likely. Then have a set of things you can check to see whether that’s the issue or not. Sometimes that might be multiple things that need to be checked separately to give confidence.

Is the chip broken? Is it powered? Is there a broken joint? Is there a solder bridge somewhere? is there a solder bridge somewhere I can’t see like under the chip to the power tab? Is there a broken trace or broken via? Is there a trace shorted to 0V or an adjacent trace? Is there a trace shorted to something that looks similar on a scope but is still wrong? Is the chip the right part number? Are there any other chips that could be wrong that could cause this like the shift register? Could the chip be counterfeit or a QC failure? Does the chip look different to the others? Does replacing the chip or swapping it with a working one from the same board change anything? Are there passives around that could cause this like series resistors or shunt resistors/capacitors? Are those passives correct?

‘Smoking gun’ type checks - Is it running hot? Any damage to the IC itself? Cracked housing, bubbles, weird smells etc.
Soldering/mounting - Visual inspection under magnification, reflow anything dodgy
DC checks - Multimeter/scope probe and check that all steady state lines are where they should be. Voltage rails are up, enables are asserted, inhibits are deasserted. Especially anything that’s an output like a fault line etc. Looking for voltage correctness and no ripple/glitches/variation.
Basic AC/operational checks - Anything that can be easily just set going and tested with a single scope probe. Comms lines if you can just spam data to/from them. Check UART Rx and Tx, doesn’t need decoding, just looking for it to ‘look like data’. Check at the end of each circuit, so ESP32 out to stepper driver in would be checked at the stepper driver end. Stepper driver out to ESP32 in would be checked at the ESP32 end, or shift register end or whatever. Voltages are correct, edge rates looks like what you expect, things look consistent.
Combinational checks - Things that need multiple probes and to check timings. Things like chip select lines, clock lines, data lines. Things that could look good in isolation but need to operate with correct relative timings. State transitions all look completed at the correct times. Turnaround time from receive to transmit looks the same as for other chips that are working, etc.

Basically everything that you can think of that verifies that the actual hardware itself is ok. Replace the chip at some point here and see if it changes.

Once you’re confident the basic hardware is performing correctly, the rest has to be on a more logic/application level. That’s when logic analyzers, manually decoding data packets, logging stuff to look for errors etc. starts to come into it.

Basically, it’s no different to any other form of troubleshooting. You’re trying to figure out all the ingredients for operation and then checking them all. Ideally, you work in terms of a combination of likelihood and ease of verification. Start with likely + easy. Things like power supply rails being up, broken/poor solder, solder bridges etc. Then go to unlikely + easy. Things like broken traces, hairline shorts. Then go to likely + difficult. Things like solder issues you can’t see like tracking under the chip, broken chip, fake chip etc. Then lastly unlikely + difficult, things like timing issues, driver/firmware issues, etc. If you get to the end of that and it still doesn’t work, it has to be something else you’ve not thought of, a check that hasn’t been completed correctly, a check that doesn’t prove what you thought it did, etc. so a 2nd pass, checking things in different ways if that’s an option. Probe a different point, use a different inspection method, etc.

Similar to an engine. You need air, fuel and spark. Is the engine sucking air in? Is it getting fuel to the carbs/injectors? Is it sparking? Once you’ve done that, the next more complicated things like compression test, injectors actually firing, carbs actually letting fuel through, spark timing. Once you’ve done that, on to the more funky stuff, leakdown tests, cam timing, sparkplugs fouling, firing order, etc. etc. 90% of the time it’s no spark or a clogged fuel filter, but sometimes it’s the cams being out 180 degrees or the timing order being read in the wrong direction, or a spark plug cap that’s wrong and doesn’t actually fit onto the spark plug when it’s in place, etc. etc. so you start with the simple stuff and then go ‘well, it’s not that, what other wacky ideas can I test?’, etc. etc.

1 Like

That’s a sign you need better test equipment, not necessarily that you were being dumb. Test equipment like this shouldn’t be brittle or need babying, it’s for testing stuff. Stuff happens.

What were you using?

It was a cheapie I got years back, I already tossed it. It served its purpose I only had it out because it was in a box for so long. Normally I just use what comes in the kits so everything is as close to real as possible.

2 Likes

It’s worth having a proper bench power supply while testing stuff. It’ll tell you how much current is being drawn and you can set a sane current limit to avoid damaging stuff if there’s a short or something wrong.

2 Likes

This! Being able to look at a device under test and seeing that this particular board is out of family in some way can be a big help. For example if it draws notably more current or notably less.

Plus, having a power supply that will respond gracefully to limit violations is really helpful. Being able to set over/under voltage limits, configure foldback versus overcurrent shutdown, etc.

That was the best thing about it, watching the current draw change as things turned on and off…okay maybe I will get another one.

1 Like

Nothing quite like having a bunch of friends on the internet helping you spend your money :rofl: We are all bad influences :money_mouth_face:

3 Likes

Sorry for the lack of updates.

I can not find what the issue is on the boards I have here. Next step is to start pulling of the rest of the drivers one at a time. For now, I am just going to assume it was a bad batch of drivers. I will keep poking around as time permits.

I have been testing the crap out of every board I ship, more than usual, nothing so far. I am also now into a new batch of boards, so fingers crossed that does not happen again. I will keep updating my test procedure.


As for the input popping I am okay assuming static. For current users I am going to add some pictures of how to stow the probe and clamp to keep them in contact while not in use. From here on out I have a new upper tool mount to store them touching. I will add pictures to the build and jackpot sections of the docs as soon as I release the upper tool mount.

5 Likes

Okay, I have been really worried about how much we deal with Jackpot issues. 99.9% there is no actual issue.

I am pretty confident now that the difference is people can easily get to so many things. Ramps - SKR, I flashed them and they never plugged them into a computer, or touched them again. There were no issues because they pretty much could not mess them up. Now they can delete the config and it looks like a dead board. The issues are ranging from power only USB, to fully erased boards.

Luckily the fix is easy.

I am going to stop stressing about this so much, besides the few random drivers on that last batch every other issue is pretty much self-imposed.

I think I am going to try to work out some sort of troubleshooting flow chart for this, unless you guys have some other suggestions.

5 Likes

I think this would be a GREAT idea!

1 Like

Something I noticed that Bart is doing with his boards is the USB does not supply power to the ESP. You have to supply power seperately. The USB is data only. That way you can’t power up without energizing the steppers.

This would be a good thing for new users and yourself.

Yeah, definitely. There’s a reason first line tech support follows the same script. Nobody likes being told ‘can you turn it off, wait 10 seconds and turn it on again’, but the simple fact is that this is often the solution.

This could be that equivalent. Automatic response to anyone asking for support: Have you followed this flow chart? If not, please do so. If so, please paste the output from steps 6, 12 and 14 here along with photos of how it’s wired, etc. etc.

3 Likes

Interesting, That is a different way to handle it.

2 Likes

The cheap tiny bee I have has a jumper to enable usb boot.

1 Like

A hardware or solder jumper is super easy to add, not a bad idea. I need to think about what is best for super noobs.

I think no usb power is better, telling new people to ignore certain errors sometimes but not other times is confusing.

3 Likes

This is the key challenge that Ryan faces.
We have all kinds of users that build V1 machines, and they bring all different perspectives.

Maybe, but it sort of sucks for a user like me that wants to be able to plug a jackpot in to a computer, flash it, and then take it to a machine somewhere and install it.

I think perhaps having a jumper on board to enable/disable this would be the best option overall. Then, we can argue about the default.
I think the default would be no USB power.

2 Likes

I am pretty confident the vast majority of users only use these boards with the power supply and will never plug in a USB.

1 Like

We have another user asking about this in a thread this morning.
Maybe you should get and stock in the store a “Power lift USB adapter” (Which deletes 5V power from the USB cable) for the cases where users want to run a sender.

1 Like

We should look at how the CP2012 is powered. I’ll go look at this late today or tomorrow. We want the CP2102 powered from USB. If there was a jumper, it should be where we pick up USB power to run over to the 3V3 regulator that powers the ESP-32 from 5V.

If the CP2102 isn’t powered from USB then the chip won’t work and there’s no comms with the ESP-32 over USB ever.

1 Like