Thursday, August 13, 2009

Configuring HugePages

Hugepages is a mechanism that allows the Linux kernel to utilise the multiple page size capabilities of modern hardware architectures. Linux uses pages as the basic unit of memory – physical memory is partitioned and accessed using the basic page unit. The default page size is 4096 Bytes in the x86 architecture.
Hugepages allows large amounts of memory to be utilised with a reduced overhead. Linux uses a mechanism in the CPU architecture called “Transaction Lookaside Buffers” (TLB). These buffers contain mappings of virtual memory to actual physical memory addresses. The TLB is a limited hardware resource, so utilising a huge amount of physical memory with the default page size consumes the TLB and adds processing overhead.

The Linux kernel is able to set aside a portion of physical memory to be able be addressed using a larger page size. Since the page size is higher, there will be less overhead managing the pages with the TLB.
In the Linux 2.6 series of kernels, hugepages is enabled using the CONFIG_HUGETLB_PAGE feature when the kernel is built. All kernels supplied by Red Hat for the Red Hat Enterprise Linux 4 release have the feature enabled.
Systems with large amount of memory can be configured to utilise the memory more efficiently by setting aside a portion dedicated for hugepages. The actual size of the page is dependent on the system architecture. A typical x86 system will have a Huge Page Size of 2048 kBytes.

“HugePages_Total” and “HugePages_Free” provide information about the amount of hugepages allocated (removed from “normal” memory) and the amount free (available for use).
HugePages is memory managed in 2 Mb blocks (this is huge compared to 4 Kb). This memory is also locked in RAM and cannot be swapped out. It is not even considered for swapping.
Programs that would like to use this memory need to have additional flags. Oracle attempts to use hugepages memory for the SGA if available. There are a number of permissions associated with using hugepages as well. See next sections for details.

The huge page size may be found by looking at the /proc/meminfo :
# cat /proc/meminfo |grep Hugepagesize
Hugepagesize: 2048 kB

Follow the following steps to enable Oracle to start using Hugepage.
=====================================================================
1.Hugepage counting formula
(SGA_SIZE/2M ) + 100
So for 10G SGA hugepage should be set to ,
((10*1024)M / 2M ) + 100 = 5220
You can find SGA size from show sga command on sqlplus prompt.

2.Please put following kernel parameter into /etc/sysctl.conf file.
vm.nr_hugepages = 5220

3.Run the following command.
#sysctl -p

4. Modify /etc/security/limits.conf file to have following entry.
oracle soft memlock 20086560
oracle hard memlock 20086560
Then Reboot the machine.

5.Check the Hugepage Total and Hugepage Free using following command.
#cat /proc/meminfo | grep -i huge
This should return like following.
HugePages_Total: 5220
HugePages_Free: 5220
HugePages_Rsvd: 00
Hugepagesize: 2048 kB
Also check the memlock setting using ulimit -a command.

6.Start the Oracle DB and check whether Hugepage is getting allocated or not by using following command.
#cat /proc/meminfo | grep -i huge
Issue cat /proc/meminfo to see how many HugePages are available. If the number of available is much less then the number you configured, reboot the system to defragment the RAM. Linux needs to find continuous 2 Mb pages.