Kriek's blog

Roll your own Unix or something similar

2012-01-06T13:51:00.000Z

I came across this very interesting tutorial by James Molloy on how to roll your own Unix like clone, with nice, detailed, step by step instructions. It assumes your development environment is going to be a GNU/Linux environment.

It's aimed at developing a *nix like operating system on x86 architecture. All the way through setting up a development environment, boot loader, interacting with the screen, dealing with interrupts and the timer, user mode, etc.

This tutorial can be used as a guide for creating any operating system, or simply boot programs, for x86 and x86-like architecture, for example the RDC CPUs used on devices like the Bifferboard.

Introduction to VMWare vCenter Operations

2012-01-04T14:16:00.000Z

I'm busy looking at VMWare vCenter Operations Enterprise. I've never dealt with it before so it could be interesting. I'm going to give an overview of what this product is, where it comes from, and what it tries to achieve. I'll also give some first impressions as a user of it.

It appears that VMWare bought a lot of products in order to give their customers various comprehensive IT management capabilities for their software and platforms. Like with most big vendors buying other software products and integrating it with their own offering, I expect some quirks with the integration.

VMWare vCenter Operations started life as a product called Alive by Integrien. Integrien was acquired by VMWare around August 2010, so VMWare has had some time to assimilate the product to make it their own. vCenter Operations comes in a few different sizes, Standard, Advance and Enterprise:

The smallest is Standard which handles up to 1500 vSphere deployments, so it's not that small.
Advanced is Standard plus Capacity IQ, VMWare's capacity planning product.
Enterprise is a whole combination of things. It includes what Advanced has, plus Smart Alerting, vCenter Configuration Manager and then deep integration with major monitoring products on the market (including the open source varieties). This enables it to also include a view of non-VMWare environments, inside and outside of the organisation (e.g. physical builds and other providers like Amazon)

VMWare's strategy with this combination is to bring performance, capacity and configuration management closer to each other. They also want to make sure vCenter Operations integrate deeper with other VMWare products, which can be both a good and a bad thing.

What this software aims to give you is a powerful dashboard for your enterprise. It basically allows you to view the status and health of your applications, systems and infrastructure, help you find faults, the cause of these faults and alert you on faults. It also makes an attempt to predict faults that may happen in the future, and also goes as far as trying to show the financial impact of faults or underutilisation.

Anyone with experience of monitoring and management tools knows getting the above right is no easy task. A lot of customisation needs to be done, and the interface has to be flexible enough to be able to build the right views.

The interface for vCenter Operations is a web interface, a fairly usable one. It's very customisable, and you put together views in a portal and widget style. It also has a lot of graphing and other graphical features, making it look attractive during a sales presentation and when showing it to management. That said, in a large environment a new user will still need to be shown where to find what they are after and also need some help in setting up their dashboards the way they want.

With regards to health and thresholds, the product tries to rely on setting these by itself mostly. In a large environment, and especially in a dynamic virtualised environment, it's impossible to manage thresholds individually. The product makes note of what it believes are normal trends in the operation of a system, and alerts when there are anomalies. If they get this right, it would be very good, but this is also not an easy thing to achieve technically, so it will be interesting to see how well this works out.

Overview of Veritas Cluster Server (VCS) Configuration Files

2011-12-30T13:47:00.001Z

In a previous article, Introduction to Veritas Cluster Server, I gave a general overview of the VCS product and how it's put together and what you can do with it. In this article I'll give a general overview of how you go about configuring it.

Before writing this, I have had no dealings with with product before, so hopefully what I cover here will help some lazy folk out there to quickly get an idea of what configuring this product is about. I'm going to be looking at it from a Solaris perspective, since I believe most of the deployments of this product out in the field is on Solaris, and it's what I may have to support and perhaps you too. I think it's very similar to the way it's done on Red Hat, so don't fear.

Files

The configuration files are normally in: /etc/VRTSvcs/conf/config

There are two main configuration files:

main.cf that defines the entire cluster.
types.cf that defines the resource types.

There are more like the above, if more agents are used. For example Oracletypes.cf.

These configuration files are loaded an maintained in a specific way. The first node to start up in the cluster reads the configuration file from disk and keeps it in-memory and when other systems come online they have this configuration synchronized to them. They write these files back to disk, and also updates to it gets written back to disk this way. The only time you can really edit these from the command line is when the cluster is stopped. Then you edit it on one server, start that server up and the others after it.

The language of the configuration file would look familiar to most system administrators. You use curly braces around most things, and there are include clauses to import configuration from other files.

Blackberry's future

2011-12-30T12:30:00.002Z

Regardless of outages and platform problems Blackberry is experiencing, I believe the phone that always tends to win in the long run is the one that wins both with the user interface and durability. Early on this was Nokia, the user interface was intuitive, steps were thought through to happen in the way people would actually use it. You make a call, you get an SMS at the same time, the SMS won't stop you from using DTMF tones to navigate your voice mail, when deciding to read the SMS the default options enable you to reply or erase with the easiest to reach button. This couldn't be said for Ericsson phones at the time, where the steps that I described on the Nokia phone was a pain on the Ericsson. Durability wise Nokia also won. I've never had a broken Nokia, I've only had one Ericsson phone and it broke a few times. I suspect Nokia must have done something similar to what Apple is rumoured to do, have lots of internal designs and prototypes compete it out until they get it right. Nokia did slip up a couple of times, probably in a rush to market, for example with their first WAP phone and their first megapixel camera phone. I suspect it's because they made shortcuts with the selection and refinement process. Unfortunately Nokia got worse at it all, and Apple and Blackberry took over with smartphones.

Blackberry phones did some things good and, until the iPhone, they did them very well on the usability front. They had a decent battery life, the buttons were good for e-mail, the scroll wheel and later the ball was good for navigation. The menus weren't cumbersome and quick and easy to navigate. It still is not bad at those things, probably better than other phones with that. The user interface is simple but works well with most of the things you want to do with e-mail and things like Facebook. That said, durability wise I found it to be horrible. I've had 5 blackberry phones from work, and they were all replacements of phones that has broken while I had it. Only one of the times it broke was it totally my fault (I dropped it in the toilet), but this is offset by the fact that I rarely used it anyway (I used my personal phone more) and that it was inside of my bag most of the time so not even exposed to the environment. I am also not counting the times it just had parts swapped out like batteries or the ball. This was only over a period of 3 years.

In the end the Blackberry seems to lag massively behind on usability that new Android and iOS phones rule now. Maps are nowhere close to being as good, web browsing is nowhere close, interfacing with these kinds of apps and others are nowhere near. That said Blackberry phones are still a lot cheaper and it's also lighter on bandwidth so for many that it still makes sense for people to buy them. I can't comment on the durability of Android based phones, but the iPhone beats the Blackberry, hands down.

Apple always operate in the premium side of the market, so I'm expecting Android offerings to do well in the market below that. Both in kicking Blackberry butt, and also in converting formerly non-smartphone Nokia users to Android using smartphone users. Google seems to have copied a lot of Microsoft strategies when it comes to Android, while Microsoft was asleep at the time and didn't even notice their own strategies, so I am not holding my breath for Microsoft to regain much market unless they spend to the scale they did with the Xbox. I expect Android to do to smartphone competition what Windows did to Novell, OS/2 and many others. This doesn't mean there won't be a market for Apple products, I believe that Apple will still do well for a while, however there is no Steve Jobs any more but I think the company has one good phone left where he still had a lot of input and they'll work hard to make it good so people still have faith in the company and stick with them until the experience, or cost, starts to suck.

After that, is anyone's guess, but I doubt Blackberry will be one of those unless they have a very good shakeup, a shakeup that Nokia didn't manage to pull off successfully. I'm not holding my breath.

Introduction to Veritas Cluster Server (VCS)

2011-12-28T14:34:00.000Z

I somehow managed to escape having to deal with Veritas products over the years. Now I have been tasked to deal with some of it, so I'm taking this opportunity to familiarise myself with it. The product I'm looking at more specifically is Veritas Cluster Server, which is a different product from their popular file system and backup products. Veritas is now also a Symantec product, but I'm looking specifically at the shape of the product before the Symantec acquisition, because it's the form I have to support. Hopefully the details I cover here will help others who also end up having to support a VCS installation.

Description of VCS

Veritas Cluster Server is software used to facilitate and manage application clusters. Examples of these would be to manage an Oracle database cluster, a web application stack, or handle the clustering of Veritas file system products, or various combinations of all of these in a global operation.

I have been told it's a good product, but also quite expensive. It's more focused on managing a cluster for high availability than for performance. In other words its main purpose is to detect failures and perform failover operations, reliably.

How it works

It is run as a service (or a set of services) on top of the operating sytem on each server. It has its own heartbeat and synchronization system communicating over the network at layer 2 level, wanting several redundant network links to do this. It provides service over a virtual IP on the system node which is currently active. In other words clients only connect to the virtual IP, and VCS makes sure that something is available to provide service over that IP address and does all the failover magic in the background to achieve that.

It is also configured to know the dependencies of resources on each other, and will shut down and start them up in the most optimal order. For example to start up an application, it will make sure that the file system resource is brought up first, then the database, then the application. It will also make sure the network is up before making sure the IP address configuration is brought up. It also does this optimally so it will make sure that certain services can be started in parallel where possible.

Service Groups

Systems can also be grouped, in Service Groups. These are used to group systems that form a particular service. For example a bunch of web servers and database servers are in one group, and when there is a fault on one of the database servers the entire lot is failed over to another set of web and database servers. Failover can also be done for maintenance purposes. Service Groups can be in active-standby configurations (called 'failover'), active-active (called 'parallel') or a mix of these (called 'hybrid')

Resource types and agents

To shut services down and bring them up, VCS interacts with the system through commands supplied to it. You have to define these in the configuration, with defined stop, start and monitoring procedures. These are coordinated by agents. Bundled Agents, Enterprise Agents and Custom Agents exist for the various resource types. A resource type could be, for example, a database, web server or file system service, e.g. Oracle DB or an NFS server. On initial startup, VCS will determine which agents are needed to manage the services, and only those agents will be started. Each agent can manage multiple services of the same resource type on a system. For example the Oracle agent can manage multiple Oracle databases on one server.

Agents have entry points, which are usually points perl scripts are triggered to perform certain functions. It doesn't have to be perl, extensions can be developed in C++ or bolted on using other scripting languages. There are various entry points: online, to bring a service up, offline to shut a service down, monitor to check the status of a resource and other entry points such as clean, action and info.

Daemons

There are three main daemons, and one module, run on each system, that makes up the VCS service.

High-Availability Daemon (HAD)

This is the main daemon controlling the whole show. It's typically referred to as the VCS engine. It maintains the cluster according to the configuration files, maintains state information, and performs all the monitoring and failover needed. It runs as a replicated state machine, so on each node it contains a synchronized view of what's going on in the whole cluster. The replicated state machine is maintained through the LLT and GAB daemons.

Low Latency Transport Daemon (LLT)
This daemon is a low latency, high performance replacement for the IP stack, for cluster maintenance. This is done over a private network and requires two independent networks between all the cluster nodes for redundancy, and to be able to tell the difference between a system failure and a network failure. It has two major functions:

Traffic distribution - it spreads internode traffic between all the private links, for speed and reliability.
Heartbeat - This is used by the GAB daemon to determine the state of cluster membership.

Group Membership Services/Atomic Broadcast (GAB) Daemon.

This does two things:

Maintains cluster membership, setting nodes as up or down based on heartbeat status.
Handles cluster communications, doing guaranteed delivery of point to point and broadcast messages to all the nodes.

I/O Fencing Module

This makes sure that only one cluster survives a split of the private network. It determines who remains in the cluster and makes sure that systems that aren't members of the cluster any more can't write to storage.

Other Processes

Veritas Cluster Server comes with a couple of other commands and processes:

Command Line Interface - to manage and administer VCS.
Cluster Manager - This comes in two forms. One is a Java based graphical user interface, the other is a web interface.
hacf - This is a utility that can verify the configuration file or make HAD load a configuration file while running.
hashadow - This watches the health of HAD, and restarts when needed.

Cluster Topologies

VCS supports a lot of different cluster topologies, this is where you can start to see the value and strength of the product. It supports from the most basic topologies up to fairly complex, and useful configurations.

The most basic form is the asymmetric, or active-passive setups. This is where there is one live server that runs an application, and there is another server that can be started up and failed over to when needed. Then it can also support symmetric, or active-active setups. Here you can have one server with one application, and another server with another application, and when one of the servers goes down, the application on that server gets launched to run on the other server along with the application already on there.

The possibilities get better from here. For example you can have multiple servers sharing a few spares, banking on the fact that not all of them will fail at the same time so you can get away with only a few spares. Another is that you can have a bunch of servers running multiple applications each, and if one of them fails it can shuffle them around on the remaining servers that has available capacity to run the application on. It can also handle failover between data centers, for example for disaster recovery. Neat.

Configuration

I think it's best that I cover configuration in another article. So, I've done that in the Overview of Veritas Cluster Server Configuration article.

The Media Causes Peanut Allergies

2011-01-29T15:47:00.006Z

Anyone who has children probably know a few other parents who claim their children are allergic to certain kinds of foods, including peanuts or nuts. It also seems this is becoming more common. Is this really so? I don't believe so, and I blame the media a bit for making it seem like that.

Here are some interesting bits of trivia:

25% of families believe their children have food allergies. 4% of them actually do.

Peanuts are legumes and the other tree nuts are dried fruits. Allergies to them are different, but it's not uncommon that a hypersensitive person is allergic to multiple things.

About 3-4% of people have reported food allergies. Most of them to certain fruits, then vegetables, then milk, then seafood, then latex, then tree nuts and only 1% of them to peanuts.

The most common reaction to a peanut allergy is eczema (40%) hoarseness (37%) , asthma (14%), anaphylaxis (6%), digestive problems (1.4%).

According to current statistics about the same amount of people get struck by lightning per year in the US as people who get anaphylaxis from peanut allergy.

Yet the Food Allergy Initiative say: "Peanut allergy is one of the most common, serious and potentially fatal food allergies."

Only 13% of severe cases of allergies to foodstuff are to people over the age of 17.

Most of the studies indicating an increase in peanut allergy in the US didn't include an actual allergy test. The widely report study in the UK that shows the occurrence of it has gone up from 0.5% to 1% in the UK was not considered 'statistically significant', especially considering the small sample size. It was obviously not insignificant enough not to make headlines all over the papers.

In other words, adults who claim they have a peanut allergy that will cause serious anaphylaxis, or digestive problems, are extremely rare.

What makes it hard to study it properly is that people with peanut allergies are quite rare and it is quite hard to test reliably. Because of the remote risk of it being fatal the only true test by oral challenge (making them eat it, placebo controlled) is not always done, about 40% of people who respond to blood or skin tests don't actually show symptoms when eating it.

People have the right to believe whatever they like about their own allergies, but the sad news is that according to another study, children who were told that they were allergic to peanuts had more anxiety and felt more physically restricted than children with diabetes.

Precautionary principle and vested interests

2009-12-23T15:26:00.013Z

I'm noticing some interesting parallels between two very controversial topics normally supported by very different camps. The one is invading, or liberating Iraq, the other is action over climate change.

The case for invading Iraq was put forward by politicians. They portrayed it as if they were acting based on information from the experts in intelligence and military employed by various government funded institutions. The threat was Saddam Hussein and the notion that he had weapons of mass destruction. This threat had been established by the experts based on evidence. The evidence wasn't open to the public eyes for various reasons. One reason was the fact that it's sensitive information that could compromise the security and intelligence sources of many nations. Another reason is that it's not necessary to release it, since various experts in the field of intelligence, military and politics have had a look at it and they are the experts who are better qualified than the public over this matter.

The case for implementing treaties between nations in order to force a curb on carbon emissions were put forward by politicians. They portrayed it as if they were acting based on information from the experts, the climate scientists employed by various government funded institutions. The threat is that if we keep on developing as we are, with transportation, heating, manufacturing and electricity running on fossil fuels from oil, gas and coal, we are going to pump so much CO2 into the atmosphere that it will heat up and change the weather catastrophically. It will significantly alter the climate and increase the effects of weather based natural disasters, droughts and displace people because of rising sea levels. This threat has been established by thousands of scientists working for various government funded institutions and by the IPCC, an organisation established by the UN for the purpose of studying the risks of climate change caused by human activity. Some of the data they work with is not completely open to the public, and they've had various reasons for not sharing it, but it doesn't matter since all the experts agree and they have peer-reviewed each other's data.

The case for invading Iraq and toppling Saddam was based on the opinions of the experts, even though they didn't have conclusive proof to show to the public, and Saddam was accused of keeping it secret. The precautionary principle was put forward for action. The logic is that even though they can't prove beyond reasonable doubt the level of threat that Saddam posed to the region or the rest of the world, the risk of doing nothing is that he could use these weapons and expand his power and end up doing terrible things to the world. These things could be comparable to the atrocities of the world wars less than 100 years ago. We were told that we have to invade now to ensure this won't happen, we can't just 'do nothing'.

With global warming the same precautionary principle is used. It is said that if we do nothing, and follow the 'business as usual' approach, the potential effects are catastrophic. Doing something about it, they say, is like buying insurance for your house.

We know with the Iraq war, doing something about it cost a lot. We actually don't know how much it has cost yet in total. I believe we'll learn even more lessons in hindsight regarding the cost. The cost was not only money, but also the respect the world lost for the US and the UK which will constrain them diplomatically in the future. It certainly cost a lot more than what they told the public, and likely more than they believed themselves. The fact that the US economy was going through a massive boom on the back of low interest rates at the same time as it was fighting a very expensive war might teach us more about other causes of the recession.

After the Iraq war, it now appears that Saddam didn't have weapons of mass destruction. Unfortunately we'll never know for sure, especially since the person who is the most likely to know, Saddam himself, has been executed. Even with that, we still can't say conclusively that it was the right thing to do or not, it's still open for debate. What we do know is that hundreds of thousands of people died violently, and it cost a lot of money.

There are many things we can learn from this. One is that the public often know better than the experts, even if they don't have the information the experts have. Especially if the experts, the organisations they work for and the politicians have the potential to gain power and money by stressing the issue. Going to war was never a decision that the public had a chance to vote on, and neither is action over climate change. In the UK and the US both the main opposition parties were supporting the war, or during the elections they were all promising action over climate change. The IPCC, CRU, GISS etc. are all organisations that have seen their budgets increase exponentially on the back of global warming hysteria, and people like Al Gore and Rajendra Pachauri both have serious vested interests in many companies that will make fortunes on the back of a cap and trade system.

The precautionary principle is either useless, or dangerous, if you don't weigh in the costs and all the right alternatives correctly. Especially if your data is biased. I always see the parallel between the precautionary principle and Pascal's wager. Pascal justified spending his life on religion by weighing up the pros and cons of the potential of Christian faith, where the reward is a good eternal life. The problem was that he didn't calculate in the probability of picking the wrong religion out of the number of different religions around the world. The probability would be very high that we was going to pick the wrong one, but based on the information he received in the part of Europe that he lived, his data was biased.

I believe our approach to these major issues with such low levels of certainty involved, should really be closer to buying insurance, with the cost worked out correctly based on the uncertainties of the event happening, and the uncertainties around the proposed solutions. The insurance should not cost more than the house.

For example, instead of spending a fortune on invading, damaging and then building up Iraq, it would have been cheaper to put together a treaty specifying that the rest of the world will take any means necessary to remove Saddam from power if it turns out conclusively that he has weapons of mass destruction. The evidence could have been with his first use of it, which could have been terrible. However, even if he was evil and insane enough to wipe out Israel in one go, he would have been hesitant to have done it knowing the majority of the world is an allied force that will wipe him out in retaliation. I also believe such a deal would have had a better chance to get support from the entire UNSC. I also don't see how you could distinguish the value of the risk of Israel being hit first compared to the value of the risk of the people of Iraq dying in the efforts to remove their dictator. It also would have been cheaper to make sure the world is armed better to fight off this potential threat if it happens, than using the ammunition on the people and infrastructure of Iraq.

With regards to climate change, let's pay for insurance now in the form of investing in things that will help us mitigate the effects if it happens. Weather related natural disasters affect poor people more, so let's make sure poor people are richer instead and make sure a richer world is ready to withstand any other changes in climate. We know, even without man made influences, the world has suffered a lot of natural catastrophic climate change events and it has happened quickly without warning in the past. I feel it's cheaper to prepare for it, and safer, than to try eliminate one of the potential causes of it happening. Just like before the Iraq war, we aren't able to predict the real effects of our action in advance. The risk of not being able to predict the cost of these actions are not like buying insurance at a price that makes sense.

E-numbers are not Evil-numbers

2009-08-25T17:57:00.008Z

The E-numbers system is a European system to abbreviate various substances included in food products, and appears on the packaging. It seems like people are naturally suspicious of these codes, probably because they appear to indicate something very artificial and we would rather trust something that's more natural. For some, it's more than just a suspicion, which offers an opportunity to market products as being 'Free from E-numbers'.

I suppose the original purpose of E-numbers was to actually allow product manufacturers to use less printing space on their packaging to list the ingredients, and to eliminate ambiguity. This is particularly useful when printing information on small or multi-lingual packaging. Unfortunately it seems like this effort to help the consumer be more informed has backfired.

E-numbers are intended to be used to list any ingredient that has an E-number, but commonly used to list additives with long chemical names. Some of these can indeed be harmful, while some people are allergic to others, but many, or even most, are not only perfectly safe, but something you should actually want. The E-numbers in the range E200-E299 are preservatives, including sulphur Dioxide (E220) and carbon dioxide (E290). The range E300-E399 are antioxidants, including Vitamin-C (E300) and Vitamin-E (E306) and so on. These are chemicals found in many natural sources like fruit and herbs.

In other words, all sparking mineral water contains E290. In fact, a sparkling drink with vitamin-C would contain both E290 and E300. Many organic food products contain E-numbers, it's just never listed like that. In organic products, the full name of the plant material is used, even though the plant can contain dosens of substances which have their own E-numbers. Many natural ingredients derived from natural processes like fermentation have an E-number (e.g. xanthan gum). Even many of the popular food colours, like beta-carotene (E160a), are also sold as beneficial supplements to maintain and improve your health, or occurs naturally in carrots.

The substances that are normally obtained in a synthesized form, are usually based on something we used to extract from natural plant materials. The synthetic process is cheaper, and contrary to popular irrationality, a cheaper process means it's more efficient and has less of an impact on the environment. In other words we don't have to grow and destroy loads of styrax trees to obtain benzoic acid any more, just like we don't have to harvest many willow trees to eat the bark, instead of using simpler raw materials to produce aspirin.

Many people believe, mistakenly, that foods, medicines and supplements that are naturally produced are somehow potentially less harmful to their health than something synthetically produced. Having some unnatural sounding numbers for these substances make it seem even worse. However, all of this shows a clear lack of understanding what the E-numbers really are, and a general lack of understanding of food chemistry, and just general irrationality.

Most of the time decisions about what we eat are not based on logic, or even on being rational. It's a very emotional thing. We base our food decisions on knowledge, but knowledge that we select emotionally. If we care about our health, and most of us for the environment too, we are influenced by information presented to us that reminds us, emotionally, of wholesomeness. Cold, hard chemicals would appear to be the opposite to us.

It would be perfectly normal to list freshly squeezed organic orange juice to contain E300. However you will probably find it hard to sell. Actually this resistance to E-numbers has gone quite far. I've just had a quick look through all the food I have at home, and so far I could only find two items listing E-numbers, and the one is probably very old stock. I only have one 'organic' product at home. Not even the Coca-Cola products list the E-numbers any more, and I remember these being the first products I noticed with these codes.

Consumer choice wins, even though consumer choice is not entirely rational, or at least not in line with my own choice. Personally I would have preferred that all the products that I'm actually allergic to list the E-numbers, instead of some leaving them out again since there is no space.

Clay on a plate

2009-05-26T23:12:00.001Z

I got a bit of clay today, put it on a plate, and made the following little video.

Some time lapse videos from Gibraltar

2009-05-16T14:26:00.006Z

I've been toying with running scripts on my digital camera, and making time lapse movies from them. I thought it would be a good way to make little videos to go with the music I make.

and a slightly older one...

Primitive understanding of money

2008-07-28T17:07:00.014Z

"Money is a man made creation and in itself worth nothing. Why are we compelled to affix price tags to everything?" I'm going to try to explain:

Money is just our convenient measure of the worth of things. It's a bit like saying gallons aren't worth anything, but at the same time more gallons of fuel are worth more than less gallons of fuel.

Perhaps some people might benefit by having a fresh look at money to realise why we measure things in this way. Some people have a very simplistic view of money and of economics in general. The kind of view you develop when you're a child in your parents' house.

Your mother is the source of a cake and other such treats, and if your brother gets more of the cake it means there is less cake for you. All pocket money and decisions about freedom comes from your parents, and you grow up to realise the importance of some of the rules they have established with you when you were young, like not letting you run across the highway when you're 6 years old. You learn that money for material things like video games, music, bicycles, skate boards and clothes comes from your parents, and you know that you can't have more of these things because your parents won't give you more money for these things, and they tell you that they don't have more money to give, so you realise that this money is important and you learn that the money at their disposal is finite just like the size of the cake is finite.

After school you might go and study, where it might be paid by your parents, or paid or subsidised from tax money, you go to educational institutions filled with teachers who normally have never achieved success in the commercial world outside of academia, or only know about getting education or educating others and getting paid by their employer, the educational institution, who gets their money from your parents either directly or indirectly through taxes.

Up to this point you learn that money is important, and that the disposable amount that anyone has is finite, and that it's their choice as to how much of that amount they want to give to others or not. You know that they get their money from employers, and employers decide how much they get or they don't get and that if they want to have more they have to convince these employers to give them more money.

Then you develop the belief that those who are poor are so because someone has decided to give them less money, and there are others who have a lot of money at their disposal who can afford to give these people more money, but don't. This seems all very unfair and is a bit like your mother deciding to give your brother two thirds of the cake and you only a third. You feel that to correct this, the employer or the government has to fix it by adjusting the portion of the handouts fairly. You also develop the idea that your mother, employer or government are what controls the allocation of money, and that you need to combine the strength of everyone in the same position in order to campaign and persuade these authorities to give you more. For instance if you and two of your siblings tell your mother that you don't want to eat asparagus, and want carrots instead, your mother is more likely to comply especially if you all refuse to eat or refuse to stop screaming together. The same goes in forming unions to prevent employers from exploiting you and your fellow workers, or forming lobby groups or nationwide strikes to convince the government. You develop the belief that this is the only way to bring chance and establish fairness.

I can easily see how this thinking develops. It's a product of the way we grow up and the society we grow up in. It's hard for us to understand how this came to be. We learn about previous kings, dictators and rulers of the past, so there's always been an elite class with more control over the distribution of wealth, just like our parents had more control over our lives while we were growing up. This way of perceiving the system is familiar to us, and we see the suffering of others, just like we suffered when our mothers took away a treat, freedom, privilege or pocket money if we didn't do something in a way she wanted. We focus on those in control, since they obviously have the means, and hold them responsible for the suffering of ourselves and others.

Perhaps you can start seeing it in a different perspective if you take a society from scratch. Let's start with a small family, mother, father and three children. The father goes hunting, the mother prepares food and feed the kids. The oldest child makes sure the younger children won't kill themselves by going out too far and fall into the river or get eaten by lions. The older child has learned that his mother and father is angry at him in unpleasant ways if he doesn't do what they say. They instruct and teach him to do things in this way because they want him to know what to do to stay alive and succeed, and help them stay alive and succeed at that. The younger children learn this at the same time. No money is involved in establishing the worth here, only love and the will to survive and better their life and chance of survival. When facing external danger like predators or food shortages, they help each other at fighting, hunting and sharing the food. Within the family you have natural solidarity, or socialism.

Brothers and sisters could implement some kind of measurement between each other when it comes to the value of their acts, for example if one gets more honey than the other they might fight over it to settle that and decide everyone should get an equal amount except for the one who's hurt the others on purpose during play, but if the oldest brother gets more meat then they could all agree that it is fair since he needs to be strong in order to protect the younger children from predators. See, even now you have to put a measurement on the worth to the older brother, even in a family structure driven by love and solidarity.

Let's say you have two families like this, the father of one family is good at hunting while the father of the other family is good at finding edible fruits and mushrooms. Since it's better to live on a diet of meat and fruits instead of just meat and just fruits because of the pattern of migration of the animals and the seasonality of the fruits, the two families share with each other.

But let's say one of the families don't provide as much of something as the other one, or not all the time, how do you determine how much fruit and meat do you share with the other family? What happens when the other family just eats a lot and get fat, and only bring food to you from time to time? So you learn to trade, the families swap fruit and meat with each other, but not in equal amounts because certain factors determine their value. In winter it takes triple the effort to find fruits as it is to do hunting, so you learn that you need to give triple the amount of meat for the same amount of fruit during winter. If the hunter doesn't agree that value with the fruit picker, he could decide not to trade at that value. He could either decide to stick to meat in winter or dry some fruits during summer to eat in the winter, or both. The fruit picker wants access to meat in winter so he decides to lower his price. Alternatively the hunter could decide that hunting is still less effort and a lot better than doing the whole dried fruit thing, so he decides that during winter it's worth paying more meat for fruit. In summer he has to hunt less for fruit since fruit is cheap. Hey presto, efficiency reached, in winter the hunter works harder and ends up providing meat for his family and the fruit picker's, and the fruit picker works harder during the end of summer so that he has enough fruit to provide for both families over winter. All this happened without having to rely on a higher authority, campaigning or violence in order to make things fair and efficient.

Anyway, that's Econ101. Unfortunately it's hard for people to take that concept and scale it up to more complex arrangements, like we have today. Where the hunter subcontracting the hunting duties to younger and healthier males to do the running around, where the price of meat is affected by running out of animals in the forest, where the fruit picker starts to grow foods, and find foods that can store through a whole winter, where they develop a medium of exchange (money), one that can be stored and traded later, where governments starts to determine the value of the medium of exchange and can create fake money out of thin air and the distortion and loss of value is called inflation, where the hunter can give meat to the fruit picker in exchange for a promise that he'll be given more fruit in summer, plus some extra since he had to wait so long etc.

We have the vast and complex system as we have today. Even while it's efficient, and everyone is better off exponentially more than before, and exponentially more than other older or failed experimental systems like socialism, communism, anarcho-syndicalism, feudalism and the likes, there are still some who suffer. We still want to blame the winners for the losers.

Unfortunately it's hard for people to understand exactly how and why, even many smart people. Instead we fall back to the way we understand the system as we learned to discover it as we were growing up under our parents and while being students. Some deny the fact that it was the system we have that gave us the platform to develop all the technologies we have today that makes life so easy for us that we can sit in an air conditioned office all day and work light and socialise with people around the world over the internet and get drunk afterwards, or at the bottom of the scale make it so cheap to produce food that there isn't a democratic and capitalistic country in the world where people die of hunger.

I hope I'm doing a charitable deed by trying to help educate people who struggle to understand all of this, since the well intended solutions to try help those who suffer can actually bring terrible suffering to this world.

Net Neutrality Myths

2008-05-07T18:17:00.006Z

I'm getting a bit concerned over this whole net neutrality issue. People campaign for net neutrality legislation as if it's needed in order to save the internet. What worries me is that people don't know what they're asking for.

The public is being put under the impression that without net neutrality legislation, broadband users will be restricted by their access providers on what content they can access and practically hold them at ransom to force them to pay extra for popular services (e.g. Google, Wikipedia, YouTube etc.), and when that happens it will destroy the internet as we know it.

If that was really the case, we would certainly have something worth campaigning for. The internet is wonderful and standing up against the big evil corporations who are intent on destroying it for profit seems like a noble effort. Unfortunately campaigning to regulate net neutrality isn't, and it shows a lack of understanding of so much of the history and the nature of the internet, economics and the role of governments and completely ignores how the internet came to be as neutral as it is today.

This is my attempt to put together a guide in order to explain many of the myths driving net neutrality activism.

Myth 1: Net Neutrality laws have been protecting us but evil corporations wants it to go away.

This is about the biggest misunderstanding. First I have to make clear what is defined by the term 'Net neutrality'. Net neutrality is the principle that all traffic from all content providers should be considered equal at a broadband provider level. This has never been a law. This has never been explicitly dictated in any set of principles, apart from common sense between those rolling out the commercial internet in the early 90's. It just happens to be the way the internet turned out and is the reason why the internet turned out to be such a success. It's the result of voluntary contracts between consumers and suppliers of the services.

Some people claim that this principle existed in legislation long ago, for example in the US telegraph system, or in the early Acceptable Usage Policy of the internet which was withdrawn before commercial internet providers even existed.

Myth 2: Broadband providers will do certain evil things to us in the future.

Since there have been no laws in place to regulate net neutrality, since the beginning of the commercial internet providers we have been open to all these dark scenarios painted by net neutrality advocates. In other words no laws have been protecting us from evil broadband providers throttling our lines or shutting off access to sites at will. Certainly outside of the US no such laws existed either.

The first commercial internet providers were anything but neutral. They only offered a few protocols and a few sites, and even had control over the software that you use to access it with, and many even charged per hour. It's market demands that have driven internet service providers to be open, and neutral.

Those of us who's been around since the beginning of the commercial internet will know that some providers have actually tried to do just that from time to time, but common sense prevailed and we have little evidence of it happening in the long term. The bottom line is that for a provider to do this is bad for business.

Without naming the name of a particular company, I can relate the story of an ISP in a country that I am familiar with. I'll call it Meganet. A large media group owned and launched an ISP, leveraging the huge marketing ability they had through their satellite and terrestrial televisions channels and printed newspapers and magazines. They were responsible for a significant amount of new users on the internet in the country, and managed to gain over a third of the home user market. Much of Meganet was started from buying an existing ISP, one that was a commercial entity that was created by a government research institution that had its funding cut. The ISP they bought were failing despite having a strong marketing budget, mostly because they charged for access per hour, where other ISP's were all charging a flat rate per month. They also tried to have an advantage over ISPs by claiming that they could provide content that other providers didn't have, this strategy also failed. Why Meganet continued some of their failed ideas, I don't know.

Along with other acquisitions that took place in order to create Meganet, they also ended up hosting a lot of very popular content providers including some news sites. When they reached the point where they were a significant ISP, having more than a third of the home user market, they tried to use their position in order to bully people from other ISPs into joining them. They started to block access to some of the popular sites to users who aren't their own customers. For obvious reasons this strategy failed, this created opportunities for competing content providers hosted elsewhere. The restricted content providers lost a lot of visitors, permanently. They tried to block their own visitors from accessing the new competing sites, but simply lost customers quicker, permanently. These actions by them were the best thing that could have happened to their competitors. Common sense eventually prevailed and they reversed all these restrictions. It didn't take long, maybe a few months.

This same issue came back to bite Meganet again, when other ISPs refused to route their multicast video. Eventually after periods of stupidity the net returned to normal.

Myth 3: Broadband providers use the excuse of cost.

Some people say that broadband providers want to charge more is simply because they want to extract more cost out of the user by using the excuse that it costs more to provide them a service. It's certainly true that private companies want to extract as much money out of their customers as the market allows them to, that's the job of a private company. Whether consumers put up with it or switch to a provider that charges less instead is the job of the consumer.

Networking equipment and nationwide cabling usually have quite a high upfront cost, but after that the costs often stand still while customers have an ongoing monthly cost. This is part of how the business model for ISPs work. The alternative would have been that providers would charge you a joining fee, which would cover the cost of your portion of the cables and switches that makes up your access to their network and those they peer with. This fee would be very high, about 40-60 times as much as the normal subscription model. But after that for the next 3-5 years you would have to pay almost nothing for your access.

After 5 years practically all the equipment except for the much of the long distance cabling would have to be upgraded to cater for increased capacity, because unlike utilities like water and electricity, we keep on finding new uses for the internet requiring more capacity.

Myth 4: Net neutrality regulation is going to ensure faster speeds at cheaper prices in the future.

Unfortunately it's going to do the opposite, that's why it's so dangerous. Domestic broadband connections are cheap, but come with very little guarantees. It's wrong for broadband providers to place restrictions on the customers without specifying or agreeing it with customers. If it's in breach of the contract between the provider and the customer it's a matter for the courts. If it's not in breech, but something customers don't like, then they can change providers. That's why I'm in support of users testing the filtering policies of their providers and publishing the details. That way consumers can be aware of what service providers do and have a choice.

Most providers will give you a connection with guarantees. That's the kind of connection that most businesses that rely on the internet gets. To provide this level of service costs more, that's why business connections cost more. The contracts specify certain service levels and the provider has to make sure they can guarantee delivery within those service levels.

To force providers, through regulation, to guarantee service levels closer to that of business customers, would mean that they have to invest in the kind of infrastructure to meet that, and they would have to raise prices in order to stay in business. In other words regulation would mean that everyone should have something closer to the style of a business connection, you won't be able to get a cheap connection with little guarantees anymore.

Myth 5: Without regulation, the internet as we know it will be destroyed.

The internet has never been as we know it. It's constantly evolving. The best thing that happened to the internet was when commercial companies were allowed to connect to it and sell access to the internet. They were free to do it in many ways, and many creative businesses created the internet as we know it today, from content providers like Google, to service providers that used their cable television infrastructure to provide internet access, to network equipment companies like Cisco that made cheap and reliable IP routers.

If regulation starts to limit them from doing stupid stuff, or limit them from providing cheap solutions but with no guarantee, then it's going to limit innovation. The internet will stay the same as we know it, which is not what we want, we want it to evolve.

Myth 6: Net neutrality legislation will benefit consumers.

If regulation prevents broadband providers from providing cheap service with low guarantees, they won't be able to choose to offer those services. Customers who want that kind of service will have less choice. We forget that not everyone wants a fast, guaranteed connection to download video material all the time, especially if it's going to cost as much as that connection would cost if it was guaranteed. Some people are happy to read e-mail, Wikipedia, some news sites and post on blogs and forums. Actually these people are in the majority, that's why broadband providers can provide a service at an almost wholesale level that's shared by many people. It's those people who are light users that allow others to enjoy fast throughput. The problem lies with when those heavy users, who only make up about 10% of the provider's customers, use more than 70% of the bandwidth.

Regulation will protect the rights of those 10% of the people, by forcing everyone else to have the same guaranteed throughput that those 10% use. That will mean the costs will have to rise to guarantee that kind of unrestricted access. This means the person who was happy to pay $5/month for an unreliable service that he can read his e-mail over will have to pay for a much better service he doesn't want.

What this really will do is kill competition. Perhaps there's a provider that wants to cater for those heavy users and offer them good deals. Perhaps there's providers who wants to offer cheap services to people who don't expect much from a cheap connection. Without regulation it's easy to provide the latter, you sell cheap connections with low guarantees to users, and build yourself a customer base. That's how new players enter the market.

With heavy regulation you take away that option, you force every provider to sell everyone a Rolls Royce when some people would be happy with a Ford. It means a new company would have to make Rolls Royces from day one cheaper than the large entrenched monopolies in order to break into the market. All this will do is protect the big players from competition, and drive the market towards entrenched monopolies. Regulation assisted monopolies are bad for the consumer.

How to pretend that you are a useful government

2008-02-11T13:13:00.005Z

I'm starting to form a set of rules of how to pretend you're a useful government, while effectively not doing much, but also not harming anything that much either.

The first and most important stunt is to look at an industry where competition is unrestricted, then make an accurate prediction as to what the outcome on a component of it would eventually be as a result of competition, and make sure it's a popular matter. Or even better, look at all popular matters and determine which ones would be satisfied by the market eventually. Then bring in laws to tell the market to do what it would have done anyway.

Where this works better is where such things are delayed because it deals with issues between different competitors, where the best outcome would be if they all collaborate to a certain degree, since competitors normally take an awfully long time to come to agreement. Examples of these are: data and voice peer/routing, mobile roaming and inter-bank transfers. These are very good bets and earn lots of political brownie points with the electorate for making this happen quicker than the market would have come around to doing it by itself. It also creates the impression that without your brilliant powers of regulation this would never have happened.

If you're even more lazy and shy of a bit of controversy, implement a law which basically mandates the equilibrium in a market segment which has already been achieved by competition. Introduce this by citing possible dangers of it diverging too far from the equilibrium. For example put a cap on the price of some goods, either a minimum or maximum, or even the price of labour. Throw in some spin about how bad it would be if these things cost so much more or people get paid so much less. In the future this will create the impression with some that you are responsible for the fact that the current conditions are because of your doing.

It doesn't matter if any of these laws are totally unnecessary in other countries, as we see with net-neutrality, even the intellectual geeks won't observe what happens in other countries with unrestricted competition and no such laws. There's always a government protected monopoly somewhere that you can use as an example of the failures of the market. It also doesn't matter if your legislation will put restrictions on the market in the long run. On top of that, the most successful players in that market will actually be enjoying some protection by your laws because it stifles innovation from small new competitors, so you'll have their wholehearted support.

Just remember to make your adjustments an accurate reflection of current conditions that work, not those that don't work or not at levels which is out of line with current conditions, or else you run the risk of getting the blame for something instead of the credit. You only have to worry about current conditions though. Whether any of your measures will fail in the long run isn't really your problem since you'll be out of office and retired by the time this happens, at least you've left something for future governments to do in order to look useful, for example if you have aspirations for your son to get into politics.

For other popular issues that you feel you want to earn more points on, but know they're not entirely economical, fund it by taxing something considered to be undesirable, best place to look is what makes regular headlines in the media. Whether it's drinking alcohol or flying in a jet, just raise taxes on it and you'll have that extra cash to spend on big vehicles and energy consuming machines that are used to recycle plastic or extra police officers to catch people smoking in pubs.

Hacking in South Africa

2007-08-01T13:45:00.000Z

This is a little blast from my past, about 11 years ago now, when I was at school and a Robin Hood style hacker for attention.

You wouldn't have thought I would turn into a right wing troll one day.

Why minimum wage is bad for the little guy

2007-05-13T09:09:00.002Z

Minimum wage and other worker protection schemes are put in place by politicians with seemingly good intentions. Even if the intentions are good, I don't believe it's good for the economy. Let me try to demonstrate:

Imagine you have lost your job and decided to start a hot dog stand. You get a stand and set it up at a busy location and hot dogs sell quite well. Business is going well, but with the money you make you can't send your kids to university, so you want to start a second stand.

First you have to save up for a second stand, or more likely you have to convince a bank to lend you money to buy the second stand. Then since you can't clone yourself, you also have to find someone to man the second stand. You now have a loan to pay off for the second stand, so you're a bit low on cash to pay the guy who mans the second stand. You make him a deal, you pay him very little, but for each hot dog he sells, he gets some money. He figures he'll take the risk, even though at worst it barely covers his bills, at the best he can do quite well. If it sucks, he can quit.

On a good day he can do very well. Also since you've given him an incentive scheme he puts in that extra effort in being nice to passers by and it's good for business. Imagine the second stand works out, both of you make money, the loan to the bank gets paid off, then you start earning enough money to risk a third stand, everyone lives happily ever after.

Unfortunately, when starting the second stand, there's no guarantee that it will make money. It is not like getting a job and knowing you'll get your paycheck at the end of the month. It's possible that the first location you try to sell hot dogs with the second stand have no one willing to buy, and you have to change locations for a while until you find a good place. Or it might be that in the entire region there's no more demand for hot dogs than at your first stand.

Imagine instead of it going well, it turned out to be a bad plan. In this case you sell the hot dog van (at a slight loss) and let your employee go. All you've lost is the loss of selling the van in a used condition along with the small salary you paid him. You haven't lost that much, a bit of saving and you've recovered. Now you can try an ice cream van, maybe the guy that worked for you before is keen on working for you on the same terms, maybe not. In the end you will find something that works, and you've created a job. You can keep on expanding and create more and more jobs.

Let's change the story. Bring in minimum wage controls and other forms of labour protection legislation. You still want to start a second hot dog stand. You borrow money from the bank for it, and you have to employ someone on a minimum wage to work for you. If the second stand works out, fine, everyone makes money and you live happily ever after.

Imagine it doesn't work out, which is more realistic because business isn't easy and you're rarely that lucky the first time with everything. This time, your employee is also on minimum wage. You also couldn't afford to give him an incentive structure on top of his minimum wage. Now there's no motivation for him to sell more since he knows he's guaranteed to get paid his minimum wage. Regardless of this, let's say the stand doesn't work out anyway, there's no one elsewhere willing to buy enough hot dogs to make it viable.

In this situation, you have to stop the venture, close the stand, and retrench the employee. He has all sorts of rights so you will have to offer him a retrenchment package. Your losses are now say a loss on selling the van, a few months worth of minimum wage, and the retrenchment package.

With minimum wage and labour controls, you've basically made the risk and the cost of opening a second stand go up significantly. This counts for all business, if you make it harder for businesses to expand then you kill job creation for the small business.

By making the risk more expensive, you're protecting the big business, since the hot dog stands can't ever grow to compete with McDonalds, but McDonalds can risk opening another branch and if it fails they can absorb the losses through their organisation.

People who defend minimum wage are either politicians who want to gain votes by it, and members of the public who's never done anything other than being an employee and at most has had to take on employees on their superior's budget. Apart from this, large companies, even Wal-mart, openly state they like to see a raise in the government mandated minimum wage.

If you look at the UK, a country traditionally filled with many small shops in the villages. This is disappearing. The English run Fish & Chips shops have mostly disappeared, it's mostly Kebab shops run by Turkish immigrants or the likes, and corner stores by Pakistanis, Chinese take aways. The rest are brands run by big companies. People are blaming the biggest supermarket retailer Tesco for killing the characterful shops on the high street.

The reason why the Chinese, Turkish and Pakistanis can operate their shops is because it's family run. That means when they start the business they employ their family members. They employ them with no contracts, no retrenchment options, and most importantly they certainly pay them less than minimum wage. They probably don't even pay them for the first few years. These small businesses can only grow as large as their family is. The few that makes huge profits while competing with large corporations, are lucky enough to take the risk on to expand and succeed, the rest will always stay small until their children has received and education and moved on to work for big corporations themselves.

If it wasn't for these families, the beer binge-drinkers would be getting their food from McDonalds after a night at the pub. The pub would also be part of a big corporation. All the corner stores will be Tesco Express. Big corporations would have won, all because everyone now obediently pay the minimum wage.

Image of the hot dog stand from the Museum of the City of New York www.mcny.org

Can we predict future temperatures?

2007-04-01T16:35:00.001Z

We often see graphs from various places predicting how the world is going to warm up over the next 100 years. Weather prediction is a complex matter since weather is a complex system and a lot of work has been done to make these predictions. A big part of this work includes the use of super computers that simulate entire climate systems and everything affecting it, including the sea, clouds, air and sun.

Source: globalwarmingart.com

Scientists have a fairly good understanding of the physical mechanisms that contribute to the weather, and fairly accurate models to predict how a particular component of this system will react to specific influences. What makes it extremely complex is how all these work together, down to each molecule around the surface of the earth.

Chaos Theory describes that small differences in an initial dynamic system can contribute to serious unpredictability later on. This theory came about during work on weather modeling because this is where the effects of it makes the most noticeable difference.

Source: UK Met office

The above graph is scoring the UK Meteorological office's predictions from 2002-2005, a total of 115 predictions for average temperature and precipitation. A score of 1 is for a 100% perfect prediction (and 0.5 would mean a 50% accurate prediction) Tmean is temperature and Precip is precipitation

As you can see from the above diagram, temperature predictions are more accurate than precipitation. It's also clear that 5-11 day predictions are the most accurate. Most importantly, 5-11 day predictions, on average, has been 35% accurate. That could also mean that it's wrong two out of three times.

Moving on to the extreme distant future of 19-32 days ahead, temperature predictions get a score of 0.05. That means one out of 200,000 times they would have the prediction spot on to the centigrade.

Predictions for future global temperatures 50 years from now show between a 1 and 2 degree warming. I believe this really has to be taken with a pinch of salt extracted from the warming ocean. We can't even get it right one month in advance.

What I learned from my day of fame on reddit

2007-03-12T19:07:00.000Z

A site of my own that I submitted to Reddit, made the front page. It has made me a Reddit rock star, and I have to share this experience with all my fans and aspiring Reddit rock stars.

A few years ago I put up a web site demonstrating the Forer/Barnum effect of subjective validation. Yes, it sounds boring and obscure. I decided to submit it to Reddit to increase the amount of participants for analysis, but to also raise further awareness of subjective validation. Oh, and to become famous.

Here is what I have learned from this experience.

Will the traffic flood your server?

Not really, I'd say, unless perhaps you're hosting it from home on your ADSL connection and have a page full of large images. In my case the average visit took about 60k of data and at the time of writing I've done under 300Mb of outgoing traffic on the site since it was released to the Reddit scene. It was also reasonably spread out, even when it was number 4 on the front page I only got about 15 new visitors per minute. Since submitting it I have had just over 8000 visitors over a 24 hour period.

Are you mega rich now?

No way. I have some Google Adwords on my page but Reddit users aren't desperate to click on ads to take advantage of the great offers from other sites. The click through rate was 0.02%. In other words, from a whole day of fame, it resulted in only one click. So much for the big house, sports car and yacht.

Does the title make a huge difference?

I guess this one is obvious, but I think it became bleedingly obvious to myself today. I previously submitted the link with the title being something along the lines of "An interesting online personality test" or something equally as boring. The trick of the test is only revealed after taking it, so I didn't want to give it away in the link description already. This made me lose the audience, because few people would actually go through the whole thing to find out what's going on in the end unless they get told that it's worth their while. The link got about 8 votes and then disappeared into the unknown.

I resubmitted the title as "Personality test gives everyone the same results, but people rate it 74% accurate." That was the summary of what it was all about and the votes came in quickly.

Does the fame propagate to other sites?

Yes. The site ended up getting hit from various other sources. Blogs, forums and other community link sites, it was even submitted to digg where it faded out of fame rapidly. What I found most interesting is how many RSS readers and community link site aggregators and portals are out there, and how popular these must be. My top referrers were:

1. Reddit (90%)
2. popurls.com (5%)
3. Google Reader (3%)
4. Netvibes (1%)

Apart from these, there was also about 60 other links from blogs, forums and other portals which would slowly end up improving the site's Google Pagerank.

Do you get a lot of groupies?

Er, no.

Anything you want to say to your fans?

Getting to number 4 on the front page of Reddit doesn't draw the kind of traffic that you would expect by getting Slashdotted. The geek audience won't make anyone rich from ad traffic. However, it still could make quite a difference for raising the awareness of your site, so if you're working on an interesting piece of technology the few thousand extra eyeballs that could discover your work can be useful.

The title makes a difference, it helps if it actually describes what people will find once they've clicked on the link. Success on Reddit would also likely translate into a much improved Pagerank so it may pay if you're actually providing something good for the long run.

Most importantly, the Reddit community decides what is hot or not, and if you don't have something that appeals to them, forget it.

CO2 from Technology?

2007-01-25T11:12:00.001Z

I don't believe that we have been looking at all the correct culprits for CO2 release. We've either mistakingly, or intentionally, left out the biggest contributers to CO2 in the atmosphere. We only focus on fossil fuels, this might be wrong. In this document I will try to explain why I think so.

Scientist appear to be in agreement that we have an abnormally high, rising level of CO2 in the atmosphere. They also seem to link this to human activity. Let me demonstrate from the graph below, which shows the carbon levels in the atmosphere over many thousands of years from Antarctic ice core data:

Source: globalwarmingart.com

As you can see from the graph above, we also have the assumption that the dramatic rise in CO2 levels over the past couple of thousands of years is due to the industrial revolution.

It is certainly true that a turning point in CO2 levels in the atmosphere happened around the start of the industrial revolution. The industrial revolution started around the end of the 18th century, where we started to use machines in manufacturing, mostly driven by burning coal.

The increased burning of fossil fuels as part of the industrial revolution was witnessed clearly in industrialised countries where it was hard not to see the air pollution.

For the sake of this argument, let's assume that rising CO2 levels causes global warming. We can also assume that the current rise of CO2 levels is abnormal for the past many thousands of years, and that this abnormal increase of CO2 levels in the atmosphere started around the time of the industrial revolution.

However, we forget, coal burning wasn't the only big change that happened during the time of the industrial revolution. Something else, something really significant was also happening, maybe as a result of it.

Just before and during this time, the human population started to grow much faster. At the end of the 18th century we already had slightly more humans on earth than ever before in history, but during the century of the industrial revolution this number doubled, and it didn't stop doubling. The following graph demonstrates:

Image source: Brian Douglas Skinner http://gumption.org/skinner.htm

As you can see, the industrial revolution happened around the time the human population growth started to accelerate significantly. Now look at this, here is the atmospheric CO2 levels for roughly the same period:

Source: CSIRO

As you can see the population growth correlates with the increase of atmospheric CO2. It's hard to deny that humans aren't responsible for the increased CO2 levels in the atmosphere. I believe we are, but not in the way we think we are.

We have machines, vehicles and power stations burning fossil fuels and we have been using them in an increasing fashion. This seems like a likely culprit. The growth of our emissions from these technologies are shown below:

Source: globalwarmingart.com

Look at the graph of fossil fuel related CO2 and that of CO2 in the atmosphere. Fossil fuels only really started releasing a lot of CO2 into the atmosphere around the late 1900s, and a significant increase after the 1950s. The CO2 graph shows a markable increase of CO2 from just before the start of the 1800s already, almost a century before the major increase in fossil fuel burning. The graph for the increase in CO2 correlates more closely to the graph of the increase in human population than that of fossil fuel burning.

Like machines, humans are also carbon fuel burning units. The average human being release around 900g of CO2 into the atmosphere per day through breathing. The current world population is sitting at around 6.5 billion people. That means we are releasing 2135 million metric tons of CO2 into the atmosphere per year. That is, more CO2 than what is released from all of the coal burning we do on earth. Shown in the following chart:

With these numbers, the amount of CO2 released by human lungs is still a lot less than the total CO2 output by all our fossil fuel burning machines. That's fine, but we often forget that we are not the only large lunged carbon burning creatures on earth. To sustain ourselves we tend to eat other animals, some of them are quite big. Beef is the meat from our most popular large animal that we like to eat. We breed and keep cattle until they are a good size before we slaughter them for food. There is an estimated 1.3 billion cattle on earth, one for every 5 humans.

I'm going to ignore the methane emissions from these bovine creatures for now, and focus on the CO2. Measurements show that the average cow release about 11kg of CO2 from breathing, per day. That means, around 5219 million metric tons of the gas per year. Put that together, and between us and our edible animals we release more CO2 than our machines, as shown here:

Humans and cattle can't be absolved from responsibility under the banner of carbon neutrality, since no mammal takes any carbon out of the atmosphere. Most of our carbon comes from plants that we consume, which are farmed on large parts of land which used to be covered by forests. It's a neutral cycle, but it's still a cycle and a large amount of CO2 still spends most of its time in the atmosphere and as population grows this amount in the atmosphere increases.

Just to be fair I'll also make a mention of the amount of methane cattle release, which is about the same as the CO2, but has 20 times the global warming effect than carbon has per volume. This means it has far more of an effect on global warming than anything we do with fossil fuels at the moment. Actually the following graph show the rapid increase of methane since the rapid increase in human population.

Source: CSIRO

Also note that burning coal doesn't release any significant amount of methane, methane is released mostly from organic processes like digestion and decomposition.

We could keep adding other sources of CO2 through respiration, but let me rather show you some data from the Kansas State University, who are mainly interested in the effects of agriculture on the environment. These numbers are million metric tons of CO2.

Fossil fuel burning	4-5
Soil organic matter oxidation/erosion	61-62
Respiration from organisms in biosphere	50
Deforestation	2
Total:	117-119

What is clear here is that fossil fuel burning makes up only a small bit of the amount of carbon being released into the atmosphere. We know the population has increased tremendously since the start of industrial revolution, but we seem to ignore this fact and focus the blame only on technology.

Please send me any feedback on what I have written here, this document is a work in progress and I would like to improve on it.