Software development is complicated, at least in cases where a computer programmer and product manager are not the same person, which is the usual setup. There is a CEO or someone else with vision who then asks the team to build it.
The main problem is planning. When you plan stuff, everything seems so easy. You have a clear picture in your mind, you see the shapes, and you expect it to be done in an hour, a day, or over a relaxed weekend. It is not that easy for the other person. You can take it for granted that tons of small or big details will slow everything down. This fact doesn't make product owners happy, of course.
There are two ways to overcome the problem. The first one is a waterfall. It made sense initially when the software development began. The idea is to plan hard to find all the details beforehand and thus have a clear and robust plan. Meaning you end up months, if not years, planning big projects, which you can throw away because the goal is outdated before even development starts.
That's why some agile framework is the most common way to handle software projects—often scrum or kanban. The agile movement was initiated by people writing code because project managers don't understand how development works, so only software developers can bring practical alternatives.
Today, product still doesn't understand it, and some developers hate it (even though they apply Agile without knowing it). But the concept is straightforward. In the beginning, you have no idea. Tons of unknowns on both sides. That's why the work is divided into short cycles. Every cycle improves the estimation of when the project can be delivered. It is not a single date but a range that gets shorter and clearer over time.
The product then can use this information to adjust backlog accordingly to needs. Maybe something is more important to do sooner? Or something less critical but easy to do to please the client in the meantime? But the most important is that everyone understands when the project can be delivered, and there are precisely two options to control it: either there is a hard deadline, and thus scope is variable, or there is fixed scope, and therefore missing deadline. This is essential: either deadline or scope is variable; both cannot be set.
Unfortunately, only a few people truly understand this. We still can often see both scope and deadline set in stone. But to succeed, the project would need extensive planning, and we are back to the waterfall.
And that's the whole point of how software development becomes not healthy. Project managers accept agile but still push for deadlines and scope. If developers agree to those terms, they need to run like crazy. Everything is always late, there is no time to do anything properly, and tech debt is increasing; due to that, everything is even more lagging, the product is furious, and developers work even harder until they cannot work anymore.
The product starts micromanaging developers and their work to ensure everyone focuses on the most critical parts. That breaks good practices and team spirit even more and harms the project long-term (if there is still something to damage). No one knows what should be done precisely anymore. Therefore product needs to be present for every single decision. In such a situation, it is typical that even the short sprints are broken as the product happily changes direction (focus) anytime.
Over time, suddenly, someone who doesn't understand development is deciding how it is done. Professional developers are ignored, and they either suffer or instead go away. Less skilled team members, on the other hand, have a hard time learning craftsmanship. Both are at high risk of burnout.
Production code is full of unfinished buggy solutions which should never have been released in the first place. There was no chance it could be done, but it was done anyway. Marketing covers it with likable design and words, but it hardly hides the actual situation from the users.
Imagine this is how aviation or health care or construction is managed. We would have one catastrophe after another. But this is how regular web development works. No wonder tons of unusable products are out there, even from big tech.
Why are we still chasing unrealistic goals?