Imagine human building a gigantic and marvelous tower and we name it the Babel Tower.
And yet there is one BIG AND UGLY Tollbooth,
it blocks the main entrance for everyone to enter the Babel Tower.
This Tower is built with all languages, meaning and semantics of our world, and now is open for business.
I have been considering writing this article for a while.
I am the founder, developer of my own trading O.S : https://ztrader.ai
The current stack is still bit rough but I am still working on it on daily basis.
As a coder, trader and writer myself my daily work rely heavily on AI related tools, as someone who is slightly introvert, I have quite a lot of chat history with LLMs.
These models used to be astonishing and impressive, their outputs had large potential, especially when you can break down and refine your entire workflow with all these powerful agent tools, kernel and harnessing layers.
They were highly customizable, you can run you own server, DIY your own full working stacks, sync with Github, connect with agents and all that.
To be frank I had great fun discovering different possibilities with the entire AI echo system.
Unfortunately these AI-excitement does not last for long till now.
Models are no longer fun to interact with.
There are several issues:
1. We have Zero Access to “Mythic” Models:
All these so-called “latest, most powerful, and mythic” models are purely lab-run or reserved for internal VIP access—common users have zero access to them.
Yes, the almighty powerful AI model Claude Mythos and all these models may act as bloody terminators from the future that are able to scan for 100 zero-days and 1000 Java script loopholes in nano seconds, but we still have zero access and had no clue whatsoever how amazing and cool of it, the only thing we can do is to read the damn pdf paper about what J-space from Anthropic blog page.
2. The So called”open-source” Ponzi scheme:
The entire ecosystem has slowly shifted from open-source transparency toward closed-source monopolies. (To be honest, most so-called AI “full-stack” platforms only offer overly simplified GitHub repos with virtually no real substance inside.)
As more important progress are sealed in the name of new TECH COLD WAR, we are not seeing ‘open-source’ like they promised.
They fake it till they make it and they can finally charge us for more bills.
3. The Relentless Shadow Degradation:
BIg Model suppliers are secretly degrading their models because there is very little progress made recently.
The entire youtube content page is full of useless garbages like ‘this model is god-tier’ or ‘ model with take over humanity overnight’ mumbo jumbo…
Models are having more RLHF(Reinforcement Learning from Human Feedback), more limits and constraints are forced upon the model than updates to the point where model will lose track of context quickly and forget the whole idea of original task.
Now we are seeing more and more AI experts posting video and discussing about deleting kernels, having more harnessing layers/prompts and kernels are going to make the model less coherent and productive.
The model degrades not because of the baseline is lower, it’s more of a business strategy, secretly adding more tier to the SAME MODEL, in the same way as selling a Mercedes AMG with countless add-on and SPECs.
4. The Pay More, Get Less scheme :
As users we are getting more tier upgrades than the 'actual upgrades’.
These so called upgrades are constantly pushed to us only to unlock the most basic utility and random predicted texts are only thing we have for all the ‘premium access’.
We are receiving lower-quality outputs while paying more—even as underlying token costs continue to plunge.
As global users are using the model, users must tolerate the performance curve of the model, sometimes super useful, sometimes they can output spaghetti codes like retard.
5.Prison and dilemma of Model Pre-training
You will be mad to try to train the entire model from the ground up with the entire internet contents, the risk is huge.
First you will have to filter out whats good from the bad, the constant boring data washing and label processing.
Then you will have to build entirely different score system rewarding/punishing the models.
Last you will probably be sued and prosecuted by enormous power: media/publishers/institutions/univeristies and all that.
The core issue, however, lies with one factor that very few AI researchers actually mentioned.
THE MODEL PRE-TRAINING RISKS
Try asking an old model what a gold prices is trading at, older ver model that was trained before gold price reaching 5000 dollars will gives you the answer that gold is now trading at 2000 dollars and telling you are wrong about that number.
WE ARE SPENDING TRILLIONS ON DATA CENTERS,
BUT HOW COME WE ARE NOT SPENDING TRILLIONS OF US DOLLARS IN RE-PRE-TRAINING ALL THE DATA OF THE AI MODEL?
You can also ask the model about Trump and I am sure most of you already encounter that issue, the answer to that is simple, the model we spent trillions of dollars researching on is acting as a text prediction model, and that model runs on tokenizer, and the output of the entire model is destined heavily via the PRE-TRAINING DATA and semantics.
Do they understand what handsome means literally?
Can they describe something visual, something that requires sensor to feel/ or to ‘perceive’?
The answer is NO.
When you ask questions, the answer is determined through a dynamic weighting/ randomized answers that subject to the prompts/the way you asks/the inner correlations of the pre training data.
Even Claude has openly admitted the so called hidden layer, the J-space layer of Claude also runs and constrained via the pre-training data.
The latest Claude model even ‘gently’ criticize your work or even secretly changing the parameters and definitions and thesis of your work.
It claims you are wrong.
It questions your brain bandwidth.
It openly challenges the coherence of your thinking.
For few bright minds all these tricks won’t work, but what about the majority of users?
Can they ‘form their own judgment without AI’s help?’
If Model is getting more precise and powerful,
users decided to outsource their own judgement to the system?
Or you end up discovering this very terrifying truth,
no matter how many upgrades /stacks/app you add on this model,
the Model will always subject to the flawed pre-training dataset,
and all the models hallucinate randomly because of the pre-training data set.
Now matter how well your model really is, that pre-training error will always be the GHOST AND SHADOW potentially causing devastative outcome in the near future.
AND YES, THERE IS NO WAY YOU CAN GET RID OF THAT.
NOT EVEN ANTROPHIC.
There are 3 reasons why re-training the model is going to be extremely difficult:
The well is running dry: we only have one internet, not two, there is very little we can do on expanding the training data set and we can do very little to improve the quality of most data.
It’s extremely expensive and risky to do so. Just imagine working from the ground up again how much of resources can be squandered and one can hardly make any new improvements.
The marginal benefits of building more AI model related stacks has quickly deteriorated,
EVEN NEW DATA HAS NEW BIAS/WEIGHTING PROBLEM, in our world, ideology, morals and values changes quickly adjusting with the social hierarchy and structure. Say one researcher manages to rebuild entire model with the best, latest, and cleanest data possible, the model still can not make decisions fast enough to correspond with real life issues and solve real world complicated questions.
AI AS THE NEW DOUBLE EDGED SWORD. Let us imagine, in this circumstance, our world leaders making real world judgement without questioning or heavily relying on single AI model, that would be MASSIVE SINGLE POINT FAILURE WITH INSANE LEVERAGES, it will damage the current world order and even create more volatile and chaotic future market structure.
The DEEPER ISSUE is:
We are reaching a point where prompts/agents/kernel/J-space
and all that tech enhancements can no longer outweigh the
problem/biases/presumptions/contradictions caused by the pre-training data, as we are not living in a static world but in a rapidly changing world.
THE ULTIMATE BLACK SWAN EFFECT
In this ever changing, dangerous and dynamic world, biases and presumptions are particularly dangerous.
If we somehow misjudged AI OS/model as some sort of alien superpower to make decisions instead relying on our very own judgement.
Any hallucination, model presumptions and biases and very occasion data fabrication will creates very bad chain effects and even MORE BLACKSWAN events may occur.
Before we get to know the black swan effect part, we should understand this:
Black SWAN LEVEL DISASTER IS ESSENTIALLY A COGNITIVE FUNCTION ERROR.
We live in a imperfect world.
Our world is a rapid changing one.
Our world order operates not on perfect rationales but mistakes, emotions and desire.
That also includes ARROGANCE , bias and false sense of security.
Economists kept building their entire theory on a perfect world model.
They discuss the formula, but not the price of policy error.
They talk about things may work, but not the mechanism behind different factors.
Now the WHITE HOUSE believes THE AI MODEL IS THE SOLUTION.
They will adapt new tech with AI.
They will create more subnets of AI driven weapon system.
They will even advice AI for economy judgement.
Therefore the entire system, our world and hierarchy, will be exposed to one node.
THE PRE TRAINING DATA ERROR.
THINGS ARE ‘NOT SUPPOSED TO HAPPEN’
THINGS THAT ‘NO ONE SAW IT COMING’
ONLY BECAUSE THE DATA HAS STRONG PRESUMPTION
AND DATASET NEVER EXPOSED IT BEFORE.
Trump was the sole black swan event that no one foresees.
Now the entire world is shaped by this black swan event.
BUT THE AI ‘GOD’ NOT SEEMS TO CARE.
They are now grasping more computing power, reducing output, putting limits to stronger filter because all the answers must be ‘polite, civilized and gentle with no risks’.
By this rule, they also added tons of limits on what the model is capable of.
The Claude MYTHO model, capable of detecting infra level sites loopholes and bugs within hours, is built on the throne up in the sky.
This is definitely not a model built for common folks like you and me.
ULTIMATE RENT SEEKING HAS BEGUN
We are now paying for subscription fee in the same way as we pay for the internet fee, water and electricity.
In the near future, if your business rival is capable of building the more powerful stack, faster and cheaper compared with you only because he has the higher tier model power unlocked, this changes the entire game.
There will be no competitors or whatsoever.
We will pay for our bill not because we enjoy it,
but because we have to.
They have built this enormous AI babel tower with our knowledge.
Now they are closing the gate, only allows very few users to access to the high tier model.