"Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"
Most of you would have encountered this error before. This means JVM's usage of heap space exceeded what you specified in -Xmx vm argument or the default max heap size (if you did not specify one).
When you get this error, before you increase your -Xmx value, make sure to profile your application and see whether the usage is justified or if there is any memory leak. See below section on profiling tools.
"java.lang.OutOfMemoryError: PermGen space"
Sometimes, you would also get a different kind of out of memory error. In JVM heap, there are different generations - Young, Tenured and PermGen. Young and tenured generations are where the regular objects from your application are stored. Size of young + tenured generation is controlled by -Xmx variable you saw above. PermGen is where JVM stores string pool, metadata about your application's classes etc.,
This error would normally happen only if your application loads a large number of classes, but is not unloading for some reason, which could be either legitimate or a memory leak. Also, make sure to check if your application heavily uses String.intern(), which could also contribute to this error.
JVisualVM provides a good insight on how your permgen grows and how many classes are loaded/unloaded at any point in time. You could also use JConsole to find this, although it is more clear and easy to find in JVisualVM. These tools only show how many classes are loaded/unloaded, but if you want to find out which classes are loaded/unloaded, you can use JVM debugging options like -XX:+TraceClassLoading and -XX:+TraceClassUnloading. They print the actual full class name which is loaded/unloaded. You can also use -XX:+PrintGCDetails and see how the permgen size grows/shrinks.
If your profiling shows that it is legimiate, increasing the max permgen size would avoid this issue. For example, this vm argument -XX:MaxPermSize=128m would set the max perm size to 128MB.
Related articles/resources on PermGen:
If you are setting max heap size or max permgen size, it is a good idea to also set min heap size (-Xms) and min permgen size (-XX:PermSize). This could avoid unnecessary resizing of the heap and might improve performance. Make sure these are set to at least what your application needs to start. This would avoid heap resizing during start up and would improve your application start up time. You could also set min and max to be the same to avoid resizing at all, but you should be very cautious and make a good choice of heap size in this case.
You could use some of the advanced performance options like -XX:NewRatio, -XX:SurvivorRatio etc., More details on other available performance options can be found here:
Before using these advanced options, it is good to profile and see what are the typical allocations in young and tenured generation for your application. This depends on the infant mortality of your application's objects i.e how long the objects live in your application. It could turn out that they are dependent on user's actions and features they use in the application. In that case, it is better not to tune them in the deployed application by default and instead profile and tune them on a per case basis, if a particular client have issues etc.,
Related articles/resources on Performance Tuning:
You could use the free tools which come with JDK - JConsole, JVisualVM, JMap, JHat, JVMStat etc., These could be found in your JDK bin directory. You could also use TPTP,
Or, if you can afford, you could use these commercial profiling tools - JProfiler, JProbe, YourKit etc.,
You could also use the debugging options in vm args to get more information. Some common debugging options are -XX:-HeapDumpOnOutOfMemoryError, -XX:-PrintGCDetails, -XX:-TraceClassLoading, -XX:-TraceClassUnloading etc., For more information on other available debugging options, see here - http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp#DebuggingOptions