Software Engineering – Discipline Is Freedom

It is strangle how bubbles go up. Early adopters obtain extraordinary results, and soon people are talking about the “chasm” being crossed, and new adopters already are being called latecomers.

One of the recent bubbles in the software industry is machine learning. It shows clearly the stage of the software industry that the job market is full of positions for developers with experience in implementing machine learning in the most farfetched scenarios. Yet, you see almost no positions for testers requiring equivalent experience. How come?

Who will be asking questions regarding the repeatability of the results? It comes as an enormous surprise to some CIOs when they uncover that several of the methods currently used in what is named “machine learning” are statistical methods, which won’t produce the same result twice, even if provided the same input. It comes as an even larger surprise that testing the implementation with the same data used for its “training” is absolutely useless, and that they need an equivalently large input dataset for testing purposes. And it surprises several developers that machine learning libraries, even if implementing a method with reliable confidence, still demand a lot of transformation of the raw input into the features used in the training process, and then a lot of filtering of the output before it is really usable.

Machine learning testing ought to be seen soon as a much desired skill. Before we start to believe that the machines will test themselves.

Upon a lot of insistence, I agreed to investigate buying a second car for those exceptional occasions when having a single car parked in my garage is not enough. My criterion was simple: I wanted to buy the smallest car that could reasonably sit 4 (not so big) people. Seeking the Internet, I’ve made a list of small cars I could buy: Fiat 500, the Toyota Yaris, Mazda 2, Mini Cooper, etc. Afterwards, I went to the concessionaries, did test drives, and compared features and prices.

Since the car would be rarely used, at least I wanted to have a little comfort when using it, and wanted at least leather seats and the ability to connect my phone to the car sound system. Not such hard requirements circa 2012. After evaluating the cars fulfilling my requirements, and having had my wife also doing a few test drives, I decided for the Fiat 500. Then I sent Fiat my data, and my acceptable conditions regarding down payment and monthly payment. I got a call saying that all was acceptable, and that I could stop by to sign the deal and get the car. My wife was happy, knowing that I take several months between starting the process and ever finishing it, and now things looked like coming to an end. Yet, when the final call comes to define the date/time to pick up the car, I warned clearly that if I arrived there and lease numbers for the car were different, I wouldn’t do the deal, and that I’m not convinced by “last minute pressure to avoid losing deals”.

About 30 minutes later I got a call from Fiat to inform me that they had some internal miscommunication between their departments. As a result, the car that they showed me couldn’t be leased in the terms they previously agreed (in the very previous call!). I could get another car on those terms… and, long story short, the call is transferred to the manager, who tries again each and every one of the oldest tricks in the car salesman book: try to convince me that the car is selling too well, that the model shown to me is too exclusive and special, that maybe I could get another car, or be more “flexible” in the financial terms, etc. Basically: it was like he was still in a 70s movie, the Saturn brand had never existed, and he was dealing with a low credit score buyer in desperation to buy a car. While hearing his speech, the only thing I could think about was how many Fiats I saw on the streets since deciding to buy one: zero!

With a service like this during the pre-sale process, I could only imagine the “miscommunications” that could happen if I ever needed post sale service. Obviously, no deal was made, and I’ve now reset the process, since I want to wait to see the Prius c. Trying to use the European sales technique in the USA won’t help Fiat. They have a reasonable car, but need to upgrade the sales technique, by a couple of decades.

There is overall recognition that the social networking bubble is about to burst. Marketing departments can no longer justify the investment in all the “social networking agencies” created by ex-employees and used under the insistence of currently employed friends to boost the “social presence” of companies in return for… nothing. Most of times, the return on investment is not measured. When measured, it usually proves that money is being thrown away without return. Yet, results are quickly hidden because nobody wants to expose that social networking marketing doesn’t provide return on investment. Better to “try again with another approach”. It is all that the economy needs nowadays: companies throwing money into ludicrous ads placed near childish comments made by adults.

When this bubble bursts, what will be left? It used to be that companies placed ads near content printed on newspapers. You can still go to libraries and see such ads. Some allow you to learn more about those times than the content of the newspaper/magazine, which carried the ideological view of the time. That “memory” was mostly lost with the radio and television. Most of the old shows replayed today are “clean” of the ads placed when they first played. Due to the myriad of network alliances and rebroadcasting agreements across the world, tv shows try, as much as possible, to be timeless and understandable by a world-wide audience. There is little or no interest on capturing the ads displayed along with the tv shows. The Internet is probably even worse in terms of capturing the history of its time. Ads are presented based on algorithms that try to target each individual. What web page to capture as a reference for the future?

When the social network bubble bursts, and having a social network account becomes the equivalent of having a fax number today, little will be left of value for the human history. Except if someone wants to add a new chapter to the series of books on “Extraordinary Popular Delusions and the Madness of the Crowds”, full of childish comments made by adults circa 2010+.

Some software bubbles are bursting, softly. Hopefully, 2012 may become the beginning of a return to normal, similar to the slow process being faced by the house market. Hopefully, people will realize that friends are not those you met once who like a silly comment you posted online. Hopefully, real friends will not like such pitiable comments, and instead send an offline message or make a phone call to the person and ask in a really interested way: “How are things going?”. Countries should run on activities that create value, not on Prozac.

Some advertisers are finally realizing that the information provided to them about social network users produces targeted ads. Yet, that doesn’t translate into a correlated increase in sales. Obviously, they have to realize that themselves: as in any bubble, it is not in the best interest of a series of man-in-the-middle to surface such information.

There is nothing more “bubbly” than the small form factor market: phones and tablets. Let’s face it: people that can live out a mobile or tablet today are only consumers of information, or producers of irrelevant posts in social networks. The most you can sell them is a movie, for about U$10.00. There won’t be investment decisions of 100 million dollars made without drilling down into the underlying information. Yet, what if you could offer them the delusion of importance, tied to a two-year contract worth north of U$2,000? Well, make them fell as important as they want, and give them the newest and more useful (or useless?!) device, as long as they keep the payments coming.

There is a problem with any business that depends on misinformation of customers: it only lasts for a while. People are realizing that not all of them make comments that will change the world, and that they can live without embarrassing themselves on social networks in return for less than 5 minutes of “recognition”. Most important is that economic failures of business models based on misinformation of customers are making investors wish for a little more than the fictitious number of “visitors” before signing the cheques funding absurd ideas. That is the key moment when a bubble starts to die: when the investors awake to the fact that they can no longer find more people to place in the basis of the pyramid scheme. The pyramid scheme of social networks and ludicrous phone contracts is finally reaching that point. Happy 2012!

It is not that I only saw new ones this year. This adds to all those I’ve been facing for the last few years.

5) Event Log entry for Windows HomeServer: Client Backup server failed at d:\wssg_src\whs_pp3\qhs\src\backup\util\sort.cpp(985). Because all customers really need is to know in which line the code failed, isn’t it?

4) The page at https://www.santander.com.br says: -2147217904-Microsoft OLE DB Provider for SQL Server-Procedure or function ‘p_IBPF_ObterUsuariosContaID’ expects parameter ‘@NU_Banco’, which was not supplied. L:[580]. Continuing with the trend, let’s inform the users in which line the stored procedure failed.

3) Message from webpage: ERROR: Possible problem with your *.gwt.xml module file. The compile time user.agent value (ie6) does not match the runtime user.agent value (ie8). Expect more errors. I like the part about expecting more errors. It gives the customer a lot of hope…

2) Microsoft Visual Studio. A fatal error has occurred and debugging needs to be terminated. For more details, pelase see the Microsoft Help and Support web site. HRESULT=0x8007000e. ErrorCode=0×0. It is not that most customers would face this. But those few developers that like me live out of really coding, love when Visual Studio goes away in the middle of a debugging session that at times took you a while to setup, isn’t it?

1) Microsoft Outlook: The attempted operation failed. An object could not be found. Was this information helpful? No. The information wasn’t useful!

Back for a while to the world of (near) real-time software applications, I’m reminded of one of the most difficult software development tasks: debugging device drivers. Mainly when there is a lot of previously existing code, and few unit tests. Yet, that wouldn’t be such a problem if I could get to a clean state quickly.

Despite several available virtualization solutions, one of the main problems faced nowadays with restoring operating systems to a previous state is not only the time it takes to boot a certain configuration, but the time it takes to shutdown a running operation system, gracefully. By now, I know about all the possible explanations (drivers, cleaning page file, antivirus interference, etc.). I can understand those explanations. What about those that cannot do it?

Are customers supposed to just accept that, after any confirmation about unsaved work, a running operating system may take at times 5 minutes or more to just “stop”? That is quite unbearable, and some of the “optimizations” I’ve read about, like disabling the cleaning of the page file, are creating security issues that most customers are likely unaware of. A shame on us, developers who allowed this to happen.

There is no way to judge someone based on a few interactions. Those are just a few data points in a big picture that could only be painted by shadowing someone 24/7, what is impossible. Yet, over time we start to slowly accumulate data, modifying the default BDC (Behavior and Discourse Correlation) assigned to someone. Such default is mostly based on past experiences. Those who in the past had mostly positive experiences with people have the tendency to assign to unknown people the BDC of 1.0: they expect new acquaintances to do what they say they would do. Meanwhile, those who had an intense and prolonged exposure to people that don’t behave as per their own talk assign to new acquaintances the BDC of 0.0. Both extreme cases are uncommon, and most people will have a “default” somewhere in the middle. Obviously, most people don’t think about this consciously.

As life progresses, one starts to realize that first impressions are almost always wrong. That abrasive person you once met in a business meeting may turn out to be the most docile neighbor you will ever have. The “nice guy” you once met, who agreed with you on almost everything, is eventually uncovered as a backstabber. The nice teacher, who everyone liked, may later be exposed as totally incompetent, and student’s grades were just being inflated to get everyone’s sympathy. You didn’t know what you didn’t know, and the “easy teacher” didn’t really help in the long term by making things “easy”. Yet, most people have a tendency to prefer the people that are “easy to deal with” instead of the people that have a greater BDC. Unconsciously, that is what I typically perceive as what most people do (again, including myself!). Consciously, I try as hard as possible to avoid that, and actively try to interact with those that take me away from my comfort zone.

There was a reason for the epiphany that once made me consider modeling such correlation between the “discourse” and the “actions” of people. Up to a point in my life, I worked mostly for small to medium size organizations. Afterwards, I was exposed to the work life of “large organizations”. I even had the benefit of already knowing a little what to expect, given that my father worked for a very large company and, at that time, a government monopoly in my country of origin. Yet, it was amazing to see in person how things worked, and how the average BDC decreases as the size of a corporation increases.

It is rare to see bad ideas flow far in large organizations. All ideas are debated, challenged, polished, and improved, up to a point in which policies and regulations in large organizations are usually great. In isolation or even in aggregation, I rarely found any “corporate handbook” in which I could easily point out something that was “pure nonsense”. Yet, when it comes to “actions”, rarely a day went by in my now decades of exposure to large corporations in which I couldn’t point out some behavior of an individual or a group that was absolute nonsense (including my own behavior when seen in retrospective!).

The book “Little Bets”, by Peter Sims, is one more of these books with a very simple premise: how to arrive at a conclusion by several tortuous paths, using anecdotal examples and ignoring all evidence on the contrary?

Not only this is one more “self-help” book, but it also has something a little more irritating than what is usually common on such books: it keeps repeating the same examples, over and over, and over, and over. After a while, nobody can stand one more reference to the examples of Chris Rock, P&G and, worst of all, Pixar. That someone could get me annoyed writing about Pixar, which is a company that I definitely admire, shows how much repetition there is in this book. There is a single sentence of Steve Jobs – “People don’t know what they want until they’ve seen it” – that is repeated at least half a dozen times.

I could accept the repetition, if the book would at least make an effort to be serious, and use its own medicine trying to question its examples and conclusions. That doesn’t happen. Suddenly, one has to simply accept that the USA military in Iraq are improvising their approach and that anthropology is their panacea. Forget about all the amazing logistics of keeping thousands of soldiers halfway through the world. Let’s improvise!

Things go from bad to worse as the examples are repeated to “prove” points, at times in inconsistent ways. It is almost as if no reviewer has really read the book from end to end. And when I was thinking things couldn’t get worse, or more anecdotal, there comes the random extrapolation about the Montessori schools to seal the coffin. Please: we can always find 5 successful people that attended any “school system”, and that proves nothing! Except perhaps that no school system is bad enough to definitely prevent success… (author: see how to challenge one’s own point-of-view?!).

At least, I had little to lose reading this book, other than my time. I got it from my company’s library, and will return it as soon as possible, to avoid having them thinking there is demand to buy more copies! Picking this book was a little bet. And I lost. Little. But a loss nevertheless.

Microsoft recently released the Microsoft Touch Mouse Sensor API, in both 32-bit and 64-bit versions. Using the SDK, developers can create applications consuming directly the 13 x 15 sensor image.

This should allow the development of innovative applications, like painting using your fingers, or recognizing gestures other than those implemented by the default driver. And it all works in parallel with the pipeline that IntelliPoint uses. Great job by the hardware and Microsoft Research teams!

It all happened when I decided to read the offer at Costco for this bright green box that claimed to “make Internet faster”. I got the Motorola SB6120 broadband cable modem for little more than U$70.00. My hope was that, if nothing else changed, I would have the modem paying for itself in less than a year, eliminating U$7.00/month that I’ve been paying to rent from Comcast a previous version (the SB5101U).

After a week of experiments, averaging speeds obtained via SpeedTest.net at about the same day during 3 different days, I got what is an impressive 52% download speed improvement, going from 13.1Mbps before to 19.9MBps after the new cable modem was installed. Upload speed went from an average of 2.8Mbps to 3.7Mbps (32% improvement). Those speeds vary considerably during the day, and cannot be taken as a reference at a different location. Yet, getting 52% more download speed for U$7.00 less per month looks like a great return on investment.