AI and Armageddon

[Ed P]

As someone who has played with AI for business purposes for nearly 25 years, I have looked at the doom and gloom forecasts for AI with mixed emotions. Please be kind on my descriptions of the following as I am now 20 odd years out of date! So beware, the following almost certainly contains errors and omissions, but if you know better please correct me!

Most of the popular press criticisms are misplaced in that automation and technology could easily be substituted for the AI bogeyman, and they are to some extent all true. Any science or technology can be abused to harm people, and automation can and does destroy jobs. While I accept the argument once put forward by Jason that tech and automation create jobs, they do so at a different pace and effect different types of people in terms of skills and education. The pace of adoption needs to be managed but unfortunately the world is ruled by techno-scientific illiterates in the Jacob Rees Mogg mould, so the gloomy prognostications might only be defeated by luck or a total misunderstanding of AI by the general public.

There are in fact many different flavours of AI, and some such as those developed using Bayesian rules are well understood and in fact form the underlying basis for most medical diagnoses. See https://www.sciencedirect.com/science/article/pii/S0933365720300774

Others such as those used in OCR or Computer Vision are equally well formed and based on well known scientific theory, so for a given input the output is easily explained.

Natural Language processing is on the cusp of being inexplicable as results are very often context dependent. English is probably the worst sinner in this respect. Just think of the problems that non-native English speakers have with words that have the same spelling but multple different context dependent meanings. e.g. “Drawing a gun” could apply to either an artist or a gun-slinger.

See https://www.cam.ac.uk/research/features/our-ambiguous-world-of-words

However the really dangerous area comes when problems are solved using neural nets. In really simple terms this is a methodology of solving a problem by listing a large number of possible parameters, then using an algorithm to find the optimum way of arriving at the solution that best achieves the desired outcome. The parameters and weighting factors are then used to produce a generalised model that best meets a range of inputs and outcomes. When expressed like this it is difficult to see the lurking dangers of this technique. The danger however comes in that for any non-trivial problem it is extremely difficult to explain how the answer was obtained.

The reason for this difficulty arises from a solution technique that uses something called “hidden layers”. Backtracking on our original ‘toy’ description of a neural net this is essentially just sophisticated curve fitting. While this may work for simple cases in most real world situations it fails, and it is necessary to apply the results of numerous neural net cases as input to another neural net (and so on). These are commonly referred to as hidden layers as their inputs are undefined until a case is run. As you might imagine this can become extremely complex and because intermediate results are used as input it verges on impossible to explain why a given result is obtained from a certain set of parameters. Although this did cause problems to those who kept questioning ‘WHY’ something happened, the problems which were addressed generally had a limited, well selected set of inputs that could be expected to influence outcomes. The problem was to some extent covered up by a bit of fast talking.

All this has now changed with the advent of so-called self optimising Large Language Models and their use of petabytes of training data (with parameters classified by very lowly paid human beings). Obviously there are hidden problems (errors and malware even) in classification, but the real problems are in the choice of parameters and in the self optimised hidden layers used to derive the solution.

It is worth looking at the scale of the problem by examining some of the published information on GPT-3.5 (not the latest model)

175 billion parameters

Neural net layers 96, each of which can have a bundle of hidden input layers.

Potentially the GPT3.5 model could be so large that no-one currently could afford the time or money to fully use it.

On the human downside, the numbers of parameters and the scope for hidden layers means that it is very difficult to know what went into an answer and whether that answer is unquestionably correct or unique.

While this may be important in areas like artistic copyright, it is not a doomsday weapon, but it might be if applied to weapon systems and the AI decides that the cleanest solution is to ‘kill the operator’; as recently happened in a US table-top simulation. Choose your scenario, but there are obvious areas where having an unknowable black-box spitting out answers could be truly the start of Armageddon or a medical disaster.

 

 

[PlaneMan]

Look at all the issues with Tesla and their self driving tech. It’s not very good in actuality.

My car has ‘phantom braked’ on me a couple of times. That is the systems thought I was going to hit something and jammed the brakes on when there was no danger.

 

 

[JayCeeDee]

I view the AI or Armageddon concept as I do life – always expect the unexpected, just be ready for it when it turns up!! As ever problems arise, even with humans when control is given to something or someone that should only be giving opinions – mmmm, back to Politics again!!

I see you’ve been keeping busy writing a novel while the site was down, Ed!😂

[Ed P]

Yes I guess it is a bit long for a post. I just get a bit annoyed at the way the press clings on to a label such as AI without really having a clue what it means. When coupled with our techno-ignorant politicians and Whitehall Mandarins this is a recipe for making pretty stupid laws or regulations, that pander to the masses rather than address the problems. (Which are usually speed of implementation and insufficient testing in a rush to market. I think rather like medication, any sensitive system should go through a lengthy validation)

Incidentally there is a way (albeit expensive) of getting an idea on how the LLM (large language model) systems get an answer. The method is to eliminate parameters one by one and study if this affects the accuracy of the model. Obviously those that have zero impact should be eliminated, just in case a rogue situation triggers them. After going through all the parameters it should be possible to clearly identify what part really influences the outcome.

Although this can show any model bias it does not really identify if the problem is in the training set. As a ‘thought experiment’ imagine a system which predicts criminal behavour in which all the criminals in the training data set are males with beards. Guess what sort of person will be fingered as a potential criminal by the AI! Validation of such training data and its parametisation (key word labelling)  is going to become increasingly important.

[Ed P]

While my post was generally applicaple to GPT type AI it would be a stretch to apply it to AI generated images. If these interest you then take a look at this write-up for Dalle-2.

Although parts are similar e.g. translating a script into a set of labels, the mechanisms for then turning this into an image montage is of course very different from a text only system.

 

[Author]

Posts