Dismantling Silos: A Path to Agile Engineering

Boundaries are necessary. That’s not the argument.

Every engineering project starts with bounding — what you’re solving, what the solution has to do, what’s out of scope. Without that, you’re not engineering, you’re wandering. The boundary is how you make the problem solvable.

The modern corporation learned the same lesson at scale. Adam Smith’s insight wasn’t complicated: split work into elements, run them in parallel, and you can deliver what no individual craftsman ever could. From Renaissance capital markets to the factory floor to the aerospace prime contractor, that logic held. Boundaries enabled scale.

When I joined the workforce in 1982, the logic was still holding — and you could feel why. I had a notebook and an HP calculator. A shared secretary supported the division manager, and before any report left the building it needed sign-off from both my branch manager and his. Not bureaucratic obstruction — that was the information architecture. Reports were dense, slow, and gatekept because they had to be. Management structure existed in large part to curate that flow — to compress what mattered, pass it up the chain, and keep the organization pointed in the right direction. The stovepipe wasn’t a bug. It was load-bearing.

Between 1982 and 2002 two things happened simultaneously that should have changed the equation. First, information handling exploded. The PC, networks, sensors — generating and moving information became cheap and fast. Second, process culture arrived. The US had watched the Japanese manufacturing renaissance and brought back a set of ideas about quality and process that got bolted onto the existing corporate hierarchy. At exactly the moment when individual engineers could span across an organization and get at information directly, the process culture locked the structure down harder.

The result in many companies: more capability to move information, less permission to use it. The stovepipes stayed. The rationale quietly expired.

I ran three programs across my career that show the delta. At SatCon on the AIPM program — Advanced Integrated Power Module, a DOE/Navy cost-share — I was simultaneously program manager and lead engineer, spanning manufacturing, electrical design, mechanical design, and simulation. We went from concept to demonstrated production-ready modules in three years on a modest budget. That approach, the sub-module test-before-integrate architecture we developed, is now standard inside automotive power electronics. Tesla uses a version of it.

At DRS, working with Allison Transmission on an integrated generator for military vehicles, we built a successful solution and demonstrated it to the Army. General officers asked why they couldn’t have more. It took ten years for the technology to gain traction — not because the engineering was wrong, but because the organizational and procurement structure couldn’t move.

At Wolfspeed, deep stovepipes. Marketing, sales, test engineering, module design, device fabrication — separate organizations, separate priorities, separate permission structures. Getting a new product from concept to release meant handing information off at each boundary and then jawboning it forward, because you couldn’t do their job for them and they had to queue the work against their own priorities. Fifteen products out the door. Every one of them harder than it needed to be.

The stovepipes were there to protect quality. They also stopped momentum.

What’s changed now isn’t the human desire to span boundaries — engineers have always wanted to do that. What’s changed is that the tools exist to actually do it. Companies that have built their information architecture from scratch rather than inheriting it — the Teslas, the newer defense tech firms — have demonstrated what happens when low-level actors have access to the full context of what the organization knows. Engineers and technicians can interrogate data, surface patterns, propose action. The information that used to require a management layer to curate is available directly. The span of control moves down the org chart.

For incumbent organizations with data already siloed, this is genuinely hard. The stovepipes aren’t just structural — they’re also where the institutional knowledge lives, and dismantling them requires executives who are willing to accept that the curation function they’ve been performing can be partially replaced. That’s not a technical problem. It’s a political one.

Christensen’s Innovator’s Dilemma describes what happens to incumbents who don’t solve it. A smaller firm with narrower scope but faster movement finds a niche. The niche gets cheaper and easier to serve. The incumbent can’t see it clearly because their whole architecture is optimized for something else. The niche expands. You know the rest.

The boundary isn’t the problem. Bounding a problem is still part of the engineering job. The question is whether, once the problem is bounded and the work begins, you’re working inside a structure that moves — or one that fills up and waits to overflow into the next pipe.

While many organizations are ‘implementing AI’ most are not working through the changes from first principles and often implementing all or nothing. The ones that don’t get around to making sure they break the stovepipes logically are going to run out of time.


This post accompanies the video Why Stovepipe Organizations Stop Working — The Unretired Engineer, April 2026.

How Physics Empowers Free Will in a Deterministic Universe

Why determinism never felt right to me — and how modern physics actually opens the door to real agency.

For years, the idea of hard determinism has bothered me. It clashes with how life actually feels. The universe as a giant clockwork machine—every particle with a fixed position and momentum, every event preordained from the Big Bang—sounds elegant in theory. But it implies that everything I’ve ever done or will do was inevitable. My choices? Just an illusion.

Hard determinists often present this view with a certain intellectual swagger, as if it signals deep sophistication. Yet many of them still look both ways before crossing the street. As Stephen Hawking wryly observed: “I have noticed that even those who assert that everything is predestined… still look both ways before they cross the road.”

That quip captures the tension. If the future is fixed, why bother acting at all? The view also carries an eerie resemblance to extreme Calvinism—some are saved, some damned, and nothing you do in this life ultimately changes the script. It never sat right with me, either intellectually or existentially.

Then I encountered the work of physicist and philosopher Jenann Ismael, particularly her book How Physics Makes Us Free. Link Her approach resonated strongly with an intuition I’d been developing for years: determinism and free will are not mutually exclusive. Physics doesn’t enslave us—it enables a deeper kind of freedom.

The “Now” Problem: Why the Instant is Trivial

Imagine the universe at a single frozen instant—the “Now.” In that timeless 3D snapshot, every particle has a position and energy vector. Past events fully determine what happens next. It looks perfectly deterministic.

But here’s the catch: that “Now” has no real existence for any actual observer. Relativity imposes strict limits. No particle (or person) can access information from outside its past light cone. At the exact moment of “Now,” that light cone has zero depth—nothing from even a tiny distance away has had time to reach you. Complete information about the universe is impossible in the present.

Laplace’s Demon—the hypothetical super-intellect that knows every particle’s state and can predict the entire future (or past)—assumes a “God’s-eye view from nowhere.” Modern physics makes that view untenable. Any real system faces data latency, noise, uncertainty, and computational irreducibility. The demon’s omniscience is a fantasy.

In short, strict determinism at the instantaneous “Now” (what I’ve called the InP, or Instant-Point) is technically true but functionally trivial. It tells us almost nothing useful about how agents like us actually operate.

Memory: The Engine of Agency

Freedom emerges not in the frozen instant, but across time through accumulated memory and structure.

Even in a blind, non-living universe, basic thermodynamics creates imprints: a rock scars the ground when it falls; waves erode a shoreline. These are primitive forms of “memory”—the past shaping the future through persistent physical traces.

Life takes this to another level. Biology is essentially memory in action. RNA, DNA, neural patterns—these are systems that record what worked and what didn’t. Evolution itself is a memory process: successful patterns persist and build upon one another.

Over eons, this scales up:

  • Simple input → output (basic matter)
  • Input → memory/comparison → internal model → action → output (living organisms)

A frog snaps at a fly. A squirrel flees at a predator’s scent. A honeybee dances to communicate nectar locations to the hive. These are not random reflexes but decisions grounded in accumulated history and pattern-matching.

Humans take it further. Language, culture, and shared knowledge externalize memory, allowing us to build on the experiences of countless others. Our decisions arise from rich internal models shaped by personal and collective history—not from some magical spark that violates physics, but from the universe’s own lawful processes.

The agent does decide. The cause of the action lies in the person’s internal identity and accumulated experience. Labeling that “determined” is technically accurate but misses the point. It’s how we function.

The Generalized Good as an Attractor

This memory-driven agency isn’t aimless. Over deep time, beings with even modest volition tend to optimize for what they perceive as “good”—survival, order, flourishing. Humans are guidable, not perfectible. We make mistakes and fall for bad influences, but signals from reality (what works vs. what fails catastrophically) are powerful if we’re willing to heed them.

History shows progress: fewer people in extreme poverty, fewer dying in wars (in percentage terms, at least). Our ancestors weren’t ignorant fools; their traditions often encoded hard-won lessons. Change isn’t inherently good, but neither is stasis. The “generalized good” acts as a global attractor, even if local maxima vary by time, place, and culture.

In deprived environments (think North Korea), external options shrink, yet people still imagine and yearn for “other worlds.” The internal model remains a generator of possibility.

My Thesis

Free will is not a violation of physics. It is the high-level, computational process of an autonomous agent using the universe’s built-in memory—personal, biological, and cultural—to steer itself through time.

Determinism at the microscopic level may hold, but it becomes trivial once you account for relativity, light cones, computational limits, and the reality of embedded agents. What matters is that you are the one deciding, drawing on your history and internal model. There is no external puppet master. The causes flow through you.

Physics doesn’t rob us of freedom. By creating a world with persistent memory, evolving complexity, and embedded perspectives, it makes genuine agency possible.

That’s why the universe feels open rather than claustrophobic. That’s how physics makes us free.

Time


Enrique Zafra

Time is fundamental, it is much of what ‘being’ is about. It is central to reality. It is central to our lived experience, it is central to our hopes and dreams. But as central as it is, it is still an enigma.

Time is a knotty problem for physics, metaphysics, philosophy, religion, something fundamental to our existence and experience but for thousands of years and billions of person hours of contemplation and analysis it escapes understanding. Like others down the centuries I find that the more I think about it the harder to grasp it becomes.

Pragmatically there is only the local now, a few moments from the past and a glance into the future. Practically there is the Past and the Future, now is just a transition from one to the other.

What is time? It seems like it is about change, and times arrow is provided by entropy, the slow winding down of the universe.

Existence, the now, is only the Plank Time instant. What stitches the universe together are memory(enabled by change) and imagination (enabled by memory.)

One option of quantum physics says that it is the conscious mind that ‘collapses’ the probability function to one reality. In that view it is our mind-memory that provides a crashing rock against which universal potentiality breaks into reality. Is it us, stitching together the universe?

Why do we talk about timespace? Because time has no meaning without space and space no meaning without time. Imagine an infinite cube of arbitrary complexity. Without time nothing about it has any meaning. You cannot travel from one point to another, there is no energy, because no movement, nothing can move, because movement is about change of location and that has no meaning with no time. Equally, without space, time has no meaning, there is nothing to change, one could say something can endure or wind down but without space for that to occur it has no meaning.

So we ‘live’ in timespace that we instantiate and make objective. It is still real in that the physics of it are fixed (probably) but is it possible that it is our (or other consciousnesses) that take possibility and harden it to reality and inflate the universe around us, out to the limits of our questing minds?

Maybe….

Expect more of ‘this.’

Xu Xianqin, Vice-Minister of Rites, overseeing the imperial civil service exam circa 1587, during the Ming Dynasty. Credit: Public domain.

Like it or not, history shows that taxes and bureaucracy are cornerstones of democracy

Article In Phys.Org by Field Museum of Natural History.

It is actually a reasonable article about a study that on its face makes sense though I believe it to be one that could be easily spun. It is a quantitative analysis of essentially qualitative factors, this is done by assigning numerical scores to identifiable attributes for multiple, ancient civilizations. You can then run analysis from simple to complex and use the scoring to make a point. Having done this sort of thing the problem is that it is very easy to bias the output especially when the results (as they all too often do) come out showing no clear message.

But as I said the basic discussion is not bad and appears balanced but the top level spin put on it by the articles title is spin. And then using images of Imperial Chinese Bureacracy and Festivals seems extremely Propagandistic. While the ancient Chinese Imperial Dynasties varied in many ways and had some good in them more by accident than intent, they were autocratic and ruthless to a one and to a fault.

This is the sort of soft propaganda one gets with things like the Confucius institute A CCP run propaganda arm, the Biden administration is giving free reign in academia to put their thumb on the scales with this sort of authorship, along with the woke fellow travelers who seem to feel that this sort of spin is necessary.

I would not protest this article, even its title so much if I felt that there was a reasonable, rational, balanced, look at the path from the past to now in modern academia especially in the west. The problem is that there are only so many minutes in a day so many moments of attention that can be paid to anything. If this sort of soft propaganda and equivalent takes up mind space, and is not counter balanced by a strong base of what current civilization owes to the West and to Industrial Anglo Sphere then the intent is to create a totally false impression of reality.

The rest of the world was not an empty bowl waiting to be filled from the west, representatives of the west did bad things. But the Christian West was always on an upward path. None of the eastern empires had advanced in any consistent and appreciable way for centuries or millennia and that had nothing to do with resources, everything to do with culture, religion, conquest and autocratic rule, none of that the peoples fault, all geography and circumstance. But pulling down the smallish bit of humanity that managed to break free and start the ball rolling uphill is chillingly evil.

CPU’s the Universe and Everything

The image from an interesting article on the ultimate in cloud computing. Hubble image of the asymptotic giant branch star U Camelopardalis. This star, nearing the end of its life, is losing mass as it coughs out shells of gas. Credit: ESA/Hubble, NASA and H. Olofsson (Onsala Space Observatory).

Seems like there must have been a mash up of astrophysics/cosmology/cybernetics a couple of weeks ago there have been a series of articles about computers and the universe. One series pointing out that once could conceive of using the AGB stars in their ‘dusting mode’ (above) as a computing engine.

But on the other side there have been a couple of articles that touch on the metaphysical (philosophical basis of reality) concept that we and our universe, are one vast simulation.

…Oxford philosopher Nick Bostrom’s philosophical thought experiment that the universe is a computer simulation. If that were true, then fundamental physical laws should reveal that the universe consists of individual chunks of space-time, like pixels in a video game. “If we live in a simulation, our world has to be discrete,”….

From: New machine learning theory raises questions about nature of science

….a discrete field theory, which views the universe as composed of individual bits and differs from the theories that people normally create. While scientists typically devise overarching concepts of how the physical world behaves, computers just assemble a collection of data points…..

From: New machine learning theory raises questions about nature of science

…A novel computer algorithm, or set of rules, that accurately predicts the orbits of planets in the solar system….

… devised by a scientist at the U.S. Department of Energy’s (DOE) Princeton Plasma Physics Laboratory (PPPL), applies machine learning, the form of artificial intelligence (AI) that learns from experience, to develop the predictions.

Qin (pronounced Chin) created a computer program into which he fed data from past observations of the orbits of Mercury, Venus, Earth, Mars, Jupiter, and the dwarf planet Ceres. This program, along with an additional program known as a ‘serving algorithm,’ then made accurate predictions of the orbits of other planets in the solar system without using Newton’s laws of motion and gravitation. “Essentially, I bypassed all the fundamental ingredients of physics. I go directly from data to data,” Qin said. “There is no law of physics in the middle…

…”Usually in physics, you make observations, create a theory based on those observations, and then use that theory to predict new observations,” said PPPL physicist Hong Qin, author of a paper detailing the concept in Scientific Reports. “What I’m doing is replacing this process with a type of black box that can produce accurate predictions without using a traditional theory or law.”…

From: New machine learning theory raises questions about nature of science

Ok so now I am going to go a bit sideways and you may want to just go on about your internet day. But while I laude Qin and his team I have a bit of an issue with what he claims re the basis is Philosophy. Not the claim that the discrete field theory sparked his concept exploration. But that the actual system he developed has anything to say about that metaphysical theory.

Taking nothing away from the team what I see seems like a straightforward application of machine learning. In fact a relatively simple one though I would laude the whole idea of applying it to physics in general. A very interesting though, like many interesting insights, oddly obvious is retrospect. (Sorry for the repeated Though clauses…I absolutely see this as fascinating insight…and possibly extremely important…it just seems like D’oh in retrospect.)

As physics is very much aligned with mathematics (I think because the discovery of each was feedback on the other) and mathematics and cybernetics are also deeply intwined it should come as no surprise that computer systems designed to create black box solutions, when fed the right kind of data, will create a black box model of physical phenomena.

The output of science are tools that allow us to predict finite things about the universe we live in, repeatably and accurately. These tools are often used by engineers to enable technologyy that make life better for everyone.

But in many ways this is an engineers (relatively narrow) viewpoint. To some large degree an engineer does not care why the tool works, only that it does and how accurately. Counter to that, a strength of the theory based + mathematical model approach is that it gives you a tool to link the rest of reality to the ‘discrete’ piece you are working on right now. A jumping off point or a linking point to other theories that allows us to move onto other problems and link the

And/But (you knew it was coming) i wonder if this has anything to do with discrete field theory per se. Maybe if the learning algorithm used had that in it this would show something of that nature, but otherwise I do not see this as showing anything in particular other than the ability of learning systems which are in some sense continuous not discrete systems to develop predictive models directly from the data (as Qin says) rather than through the labor intensive methods of theory extraction and proof that has been the basis for scientific exploration since it first evolved in the Middle Ages.

Again BUT, it has been getting harder to develop these ‘deep’ theories. Look at the colliders and other tools that physicists use these days to probe the depths of our reality. In this world there are many things, like Qin’s next test with Nuclear Fusion, where an engineering model might be much more valuable than a ‘theory of this’ if it can be captured and used in a fraction of the time.

It’s all good, fascinating, wonderful…but let’s not get ahead of ourselves.

Of ratchets and slopes, slippery or otherwise

Crossbow and the crank/ratchet for cocking it medieval period, WikiCommons

As commented on before I pay attention to Scott Adams of Dilbert fame as an interesting thinker with a fairly well defined but undefinable political gestalt. Uber liberal realist Trump supporter is maybe the best description.

One of his mantra’s is that Slippery Slopes are Not a Thing.

The following is my interpretation of his position.

A point of view/policy item with a broad ‘option space’ and supporters on both ends, say like gun control, will slide in a direction that is acceptable to the general polity (something like the Overton Window) until some point it will no longer be acceptable. Those who wish to push the policy towards one end or the other will eventually meet resistance and be unable to move the policy further ‘their way’ until some change occurs. That change may move the policy ‘back’ or ‘forward’ but it is acceptability that controls. This says that the idea of a ‘slippery slope ‘with its imagery of reaching a point where you lose control and slide to some end point it false on its face.

Having thought about this I agree with the premise in a general sense.

Two, I think important, quibbles:

1) That in a highly emotional and very dynamic situation such as one might have in the ancient Demos of Greece, or say a Constitutional Congress, a French State Committee…, the slippery slope appears to me to be a real threat. The whole of the polity is in the fight as it were and there is no stable base of opinion to dampen high flights of rhetoric and emotion. In such situations you have a tendency to move to the end state without the intermediary and if this is then enforced on the outside world the results are likely to be calamitous if the topic is one with a high degree of emotional attachment with the broader public. The Demos were tiny isolated city states and they killed a few important people and destroyed themselves but it was in the end fairly evolutionary. The US constitution was very conservative in its basis and while the result was ‘liberal’ it was not that crazy and was in line with most of the populous, plus it was a huge area with a tiny population, where malcontents could often go west if they wanted. The French Revolution was a bloody multi decadal disaster because it didn’t have any other damper than time and blood….To a large degree I don’t see this as that active other than in a Social Network Today…to some degree it explains some of the crap that goes on in odd corners of the web.

2) More important than 1) is the fact that the ratchet is IMO real. That once a law or regulation is in place it tends to create a new baseline and constituency. If the issue is fairly hot there will be pushback but in general people are for stability and a law or regulation will become entrenched. It only takes time for that to then be the jumping off point for a new effort to extend whatever policy. This may not be very logical on its face but it is a reality and is one of the reasons that any human system tends to atrophy with time. So the party who tends to desire more law and regulation have a tendency to have the edge here and they will turn the crank on the ratchet whenever they get the chance.

While England is not the US in any sense one should look at it as a bit of a case study, though the lack of the 2nd Amendment is a huge factor. A century ago guns were rare more because of their cost than anything else. Then regulation started to build up. Because of no 2A and it was very gradual there was not much push back. Today not only is any kind of firearm in private possession effectively illegal so are any edged/pointed device inclusive of scissors. The ratchet is real…the slippery slope is a thing only in very constrained cases.

We need a Base or our dreams are fantasy shading into terror.

As limited and degraded as man undeniably is, he yet remains “the proof by contraries” of the divine Prototype and of all that this Prototype implies and determines in relation to man. Not to acknowledge what surpasses us and not to wish to surpass ourselves: … is the very definition of Lucifer. The opposite, or rather the primordial … attitude is this: to think only in reference to what surpasses us and to live for the sake of surpassing ourselves; to seek greatness where this is to be found and not on the plane of the individual and his rebellious pettiness. In order to return to true greatness, man must first of all agree to pay the debt of his pettiness and to remain small on the plane where he cannot help being small; the sense of what is objective on the one hand and of the absolute on the other does not go without a certain abnegation, and it is this abnegation precisely that allows us to be completely faithful to our human vocation.

Logic and Transcendence: A New Translation with Selected Letters
© 2009 World Wisdom, Inc. pg 14, The contradiction of Relativism

OK, so I’m addicted to philosophy right now. Directed here by One Cosmos, a autodidactic philosopher and deeply anti-psychological psychologist of the once great state of California.

While some philosophers and metaphysicians can be opaque in the extreme others are as one should expect extremely good writers with depth and clarity that is remarkable. Others are opaque but readable because what you ‘get’ seems true and sensible and the opacity is often in their use of old words, Greek or Latin words to try and de conflict their meaning from the everyday words that have been so poisoned by intentional obfuscation. The above book is of near preternatural clarity type. I have found another writer, Voeglin the type that makes you want to work at it.

Our space science Economy has assets

An artist’s depiction of the OSIRIS-REx spacecraft at work at the asteroid Bennu. (Image: © NASA/Goddard Space Flight Center)

NASA’s OSIRIS-REx probe could make a 2nd stop at infamous asteroid Apophis, at Space.com, and noted in several space related blogs, eMags. This sounds like a fantastic use of a remarkable space asset.

The Japanese asteroid prospector Hayabusa2 dropped off its samples from Ryugu at Earth and is on its way for more exploration last year: Farewell, Ryugu! Japan’s Hayabusa2 Probe Leaves Asteroid for Journey Home

These craft and others such as craft like the voyagers continue to return immensely valuable data long after their primary mission is complete. One of the things NASA and other space science organizations struggle with is supporting these ships long after the original funding timeline is past. This is a great problem to have and by and large the money is found since these are very cheap deep space projects in the big picture.

So my title, the economy of ‘outer space’ is all about data, science, prospecting right now. These are valuable assets that we need to support to provide returns orders of magnitude greater than the cost in the sense of other ways of getting that data, data that is both live affirming in its fascination and valuable as part of the bedrock of our understanding of the universe.

Cheers