So a lot of arguing about AI and creative works is going on right now and a lot of it is not really all that constructive, since it seems people are often not arguing about the same things. 

I am not really here to delve into that debate. I am mostly a tech-sceptic in that I tend to assume that new technologies will end up being useful, but rarely as monumental as they are hyped up to be. And in any event there is zero reason for me to join the hype : If a new tech becomes a corner stone of our existence like the internet did, these days I am pretty content waiting for the idiot-version that actually works.

Is LLM / "AI" tech currently valuable to me as a game developer?

Well, I need to do basic internet tasks like quite a bit of email, I need to write game rules and I need to listen to heavy metal at obnoxious volumes.

Internet and email:

I answer a lot of rules questions by email. Can chat bots answer rules questions correctly? I tried a random question about patrons in Five Parsecs and it paraphrased an answer on Board Game Geek ...which itself got part of the answer wrong. 

A second test was for Rush movement in No End in Sight where it summarised the mechanic fairly accurately, then when I asked it how it got that information it corrected itself to say there is no tabletop game by that name. 

A third test was how many miniatures you need in Weasel Tech, which gave a non answer saying that you can use any miniatures you like. 

These are not scientific tests of course, but a 33% accuracy rate means I would have to go back and double check the answer instead of just taking the 10 seconds it would take to type out the right answer. And this is assuming this is not an obscure interaction where I'd have to double check the book anyways, because it is something that has literally never come up before. 

Writing game rules:

The problem here is that the LLMs basically write like a reddit idea-guy. It is often writing in very broad terms, but tends to omit the sort of detail that game rules require. A quick example asking Gemini to give me a ranged combat system for a ww2 game produces a broad overview but does not answer questions a gamer would ask, like how do I select the target, do units have to shoot at the same targets and so on. It also uses a lot of vague concepts like saying that there are modifiers but does not give exact values. The mechanics it suggested also seem to borrow from D&D and 40K quite a bit (which again fits a reddit idea-guy, not surprising since LLMs are trained on reddit posts).

Now you can continue asking questions but if I am writing this myself, I would already have answered those questions. 

This might be useful for someone who has no idea how a miniatures game works, but in that case pick up literally any rulebook or rip off Warhammer like we all did :)

You can test this for yourself pretty easily. Load up one of these systems and ask it to create you a game, then sit down to play it and see how long it takes to actually get something you can sit down to play without having to ask further questions. 

The system might have value as an idea generator. I've messed with asking it for RPG scenario ideas and some of them were not bad at all (And a few were legitimately funny), but that seems like it is just finding a way to avoid doing the fun part of writing a game. I am not sure why I would want to do that.

Conclusions?

I don't know man. I guess I am not really seeing the value for my particular niche. 

Sound off in the comments but keep it polite. I am ignoring the art portion since that is not something I am interested in.