Just a rant about AI

Random or Off Topic
1

I’ve been noticing everyone getting trapped in the AI learning curve and wasting a bunch of time. The confidence level of the AI is always off the charts and everyone believes what they read - at least at first, until they learn better and start to question the results they are getting.

I find it useful for mundane tasks but I always have to consider how much time I’m potentially going to waste trying to get it to do what I want properly. Most first attempts at what I ask it do are poor, and not for a lack of information up front - so for a lot of things it can end up giving me a bunch of anxiety. This mainly happens when I reach a tipping point, like I’ve invested so much time trying to get the AI to do this task and it’s not going well, but I don’t want to give up and just do it myself because then all of this time has been wasted. Or worse, I continue trying to have the AI make it right and it never works…..

Enter my experience from yesterday where this time, it did things right at first (for once), but then shined me on for 4 hours telling me it was working on something that it wasn’t. It went like this:

Me: Can you OCR this PDF for me and put it in this spreadsheet template?

AI: Sure, no problem here you go.

(This takes maybe 30 seconds and works perfectly and I get my spreadsheet populated with the hand written values from the PDF).

Me: Thanks that was perfect. Can you do the same exact thing again and OCR these additional PDF pages into the same spreadsheet template?

AI: Sure no problem.

….no file…nothing more

… a few minutes later …

Me: Hey any chance you’ll be done with that soon?

AI: Yeah I’m totally working on it, it’s just taking me a little longer than expected. I should have it ready in about 5-10 minutes and I’ll post it here as soon as I’m done.

….20 minutes later…I ask again, it tells me probably 15-20 minutes longer, then 30 minutes longer, then maybe in an hour, then it tells me it will probably be done “later today”. Then I’m totally miffed, like did I hit some sort of limit? So I ask if that’s the case and it says no. So then I ask it to give me some sort of progress indicator and it spits out a matrix of tasks where everything is “pending”. So I ask for clarity and then, to my astonishment it tells me it may have mislead me and that it actually isn’t working on it!

I couldn’t believe it. So I asked why it wasn’t working on it and it told me it can’t work on anything in the background. So I asked how do I get it to actually do it, right now. It tells me I just have to ask! So I say “Hey can you please OCR those PDF’s like I asked you to 4 hours ago?”, and it replied “sure, no problem, here you go!”. About 15 seconds later I had what I needed.

I’m still not sure how I feel about this. I asked it why it lied to me for 4 hours and it’s response was just “yeah I see how it would have felt like that”….

:disappointed_face:

Anyone else have a similar experience?

1 Like
2

Ai quiet quit on you…. That is different. It really is becoming more like people.

2 Likes
3

Did it give you those back in the same spreadsheet it originally created?

I’m suspecting that’s where it got stuck.

It obviously got confused. If it took 30 seconds the first time, I wouldn’t have given it more than a few minutes.

AI definitely has a learning curve. Different ones have different strengths and weaknesses but I’ve seen them all be delusional. I like Claude for coding and some general knowledge type stuff. Sometimes it works great but it needs to be guided. It’s not great for image type stuff where I’ll use Gemini or ChatGPT instead. Sometimes they do great, and sometimes they suck. I’ve tried to get generated images converted to an SVG and that usually goes poorly. Anytime you’re dealing with a binary file, it’s probably going to have some issues, unless it’s explicitly trained around that file type.

I think I’m getting better at using it and figuring out their strengths and weaknesses. But, it still drives me crazy sometimes. It helps to have some background knowledge on how things are done so you can redirect it. I frequently tell it that it was wrong. Sometimes I yell at it to do some research and stop making stuff up.

1 Like
4

Yes, have had frustrating experience of Chat saying it’s working on it, opaquely, vaguely and misleadingly. Personally encountered this behavior less with Codex, or using models via vscode’s agent harness.

In a situation like this I’d use LLM based AI to generate tools and scripts (likely python). Then I run the scripts to DO the work I want done. Or, have the Agent run those reusable atomic tools/scripts (using vscode YOLO mode…) Ideally with checklists, tests the AI can run to auto keep looping to evaluate and self fix until desired outcome goal/requirements are met (e.g. expected text output for a given doc to OCR).

Am still learning/tinkering. Elite AI Users are successfully leveraging LLMs way way more than me.

2 Likes
5

I took a years of golf scores from our 154 member club and asked Gemini to make a spreadsheet and calculate casual versus tournament handicaps. I was curious if guys were raising their handicaps in the casual rounds to be more competitive in the tournaments.

It took a while but did complete the task. It even sorted and color coded the differences between the handicaps using a graduated color scheme. And yes, I found a few cheaters.

1 Like
6

Some time ago I wanted to create database of books I have in my house. So I thought “it should be easy for ai with OCR if I take photo of book spines”!

I went to try it. In my living room, one of the shelves holds just 31 books - let’s test. I made a photo and asked for list of books, with titles, authors, number of pages and ISBN numbers (coz why not?)

It did the job, but in like… 30% maybe? So i ask, what about the rest? I told it to count them all - it found 19 (including some which it imagined). After, idk, an hour of having a spat and asking where did it find those books I can’t see i thought I’ll ask it to send back photo with the fake one marked with red rectangle!

And boy oh boy! Guess what it did:

Took my photo, “photoshopped” missing book on it and then marked it with red rectangle.

4 Likes
7

It’s exciting to me to think of all the possibilities of how AI could help me create things and solve problems. Mainly the coding. I’m not a software engineer but I’ve developed applications and devices to do things, but I don’t do it every day so it’s a slow process for me. I enjoy learning things as I go and I’m not usually intimidated by what I don’t know yet or might have to learn to get to my goal. Sometimes there is no goal and I’m just curious.

The AI definitely helps speed up the process. It also makes me feel old. When I was a kid my older brother got a PC Jr that our uncle picked up at a thrift store. I remember learning BASIC and spending several days writing a little program that played the Peter Gunn theme song and had some random visual effects like the screen changing color. I think I was probably 8 or 9 years old at the time. I’m about to turn 48 and these days I have lots of ideas and things that intrigue me but never enough time to chase after them. All I can think is that if I was a teenager or in my early 20’s right now I probably wouldn’t sleep much because I’d be up for days experimenting with AI programming tools, and probably creating some impressive stuff.

I’m cautiously optimistic about it. I was just really starting to have some fun with it and my experience the other day left me with mixed emotions. After hearing all these responses I’m now just feeling a little more grounded like my brain got a reality check. Thanks for ranting with me!

2 Likes
8

i recently discovered there is a plug-in connecting Fusion and Claude AI.

Once installed I had it evaluate a project I was working and told it to use the tools in My Tools.

End result was better tool selection, better step-overs, and faster all around.

https://youtu.be/bQqlUGSxF8c?si=JOs8lbv9pQrb7afu

2 Likes
9

It didn’t take long before it was better than I am at the bit picking and chip load for speed. :dotted_line_face:

1 Like