Setting the JAVA_HOME variable can be done in several ways.
1. Globally and at System level, i.e. no user is required to be logged in. This is the preferred way if you are configuring a server and planning to use boot strapped program that needs JAVA_HOME variable.
2. Globally and set for all logged in user, i.e. one user is needed to be logged in, to have the JAVA_HOME variable set.
3. User specific. Then the variable is only set when the designated user is logged in.
If you have different installed JDK you might find this blog useful on how to manage different JDK on Ubunut http://magnus-k-karlsson.blogspot.se/2012/03/managing-multiple-java-installation-on.html.