Legible To Machines?

Here's another short post.  I have a Chinese friend who is a software engineer.  I think her husband may also be a software engineer.  Anyway, we have been having a friendly semi-disagreement about the capabilities and impact of AI on the future of industrial society.  (It's always a good thing to have friends who are willing to disagree with you!  This helps to keep your thoughts and conclusions healthy!  She also knows more about computers than I do...)  She sees AI as somewhat inevitable due to its rapidly increasing capabilities, although she recognizes some of the potential harms that may result from the increasingly widespread use of artificial intelligence (including large language models or LLM's) in society.

As one can tell from reading some of my most recent posts on precarity, I am a bit more skeptical about the ability of AI to take over a majority of human cognitive tasks.  And in re-reading (or in my case, re-hearing the audiobook version of) Seeing Like A State by James C. Scott, I've been wondering how much of the power of present-day AI is the result of our society having been remade in order to become more legible to AI.  Scott points out the many ways in which the ruling elites of past and present societies have remade their societies in order to make them legible to the command and control organs of the State, thus facilitating easier command and control of their subjects.  Yet these schemes of legibility have often had painfully unforeseen consequences.  How have major corporate interests made our society more legible?  How have their methods also facilitated making our society more legible to large AI deployments such as chatGPT and Gemini?  What are some of the consequences we are likely to see from this re-making and its resulting increased legibility?  I wish I could ask James C. Scott such questions, but unfortunately he passed away last year.  Looks like we'll have to figure things out ourselves...