Exadata Basics

In an Oracle Exadata environment, cell node storage disks are presented to compute nodes as ASM disks, and this mapping occurs through the Oracle ASM (Automatic Storage Management) and Exadata Storage Server software (CellCLI).

Here’s a detailed explanation of how the cell node storage disks are mapped to the compute nodes:

---

1. Cell Node Storage: Physical Storage Hierarchy

On the cell nodes, the storage hierarchy consists of:

1. Physical Disks: Physical hard drives or SSDs in the Exadata storage cells.
2. Cell Disks: Logical partitions created from physical disks. These are managed by the Exadata software.
3. Grid Disks: Subdivisions of cell disks that are exposed to the ASM instances on the compute nodes. Grid disks are assigned to specific ASM disk groups (e.g., DATA, RECO).

---

2. Compute Node: Logical ASM Disk View

On the compute nodes, the grid disks appear as ASM disks, which are presented to the ASM instance. ASM uses these disks to manage storage for Oracle databases.

3. Key Mapping Steps

The process of mapping storage from the cell nodes to compute nodes involves the following steps:

Step 1: Cell Disk Creation

• On the cell node, physical disks are divided into cell disks using CellCLI.
• Example:bashCopy codecellcli -e "create celldisk all" This creates cell disks on all physical drives.

Step 2: Grid Disk Creation

• Grid disks are created on top of cell disks and assigned to specific ASM disk groups.
• Example:bashCopy codecellcli -e "create griddisk all prefix=DATA size=100G" This creates grid disks with the prefix DATA.

Step 3: Grid Disk Export to Compute Node

• The grid disks are presented to the compute nodes as logical devices. These devices are visible in ASM as ASM disks.
• ASM on the compute node identifies the grid disks through the asm_diskstring parameter, typically set to a path like /dev/oracleasm/disks/* or /dev/mapper/*.

Step 4: Discovery by ASM Instance

• On the compute node, ASM scans the asm_diskstring to identify available disks. The ASM disk header contains:
  • Disk group name (e.g., DATA, RECO).
  • ASM disk name (e.g., DATA1, RECO1).
  • Disk UUID or label for unique identification.

---

4. Common Identifiers for Mapping

The mapping relies on shared identifiers between the cell nodes and the compute nodes:

1. ASM Disk Name:
   • Assigned during grid disk creation.
   • Visible on both the cell node (cellcli) and compute node (asmcmd).
2. Disk Group Name:
   • Specifies which ASM disk group the grid disk belongs to.
3. Disk Header Information:
   • Contains metadata such as disk UUID or label.
4. Disk UUID:
   • A unique identifier present in the disk header.

---

5. Viewing Disk Mapping

On the Cell Node

• Use the CellCLI command to list the grid disks and their attributes:bashCopy codecellcli -e "list griddisk attributes name,asmDiskGroup,asmDiskName" Example output:Copy codename asmDiskGroup asmDiskName DATA_CD_01 DATA DATA1 DATA_CD_02 DATA DATA2

On the Compute Node

• Use the asmcmd command to list ASM disks:bashCopy codeasmcmd lsdsk -k Example output:bashCopy codeGroup_Num Disk_Num Incarn Mount_Stat Header_Stat Mode_Stat State Path 1 0 12345678 CACHED MEMBER ONLINE NORMAL /dev/oracleasm/disks/DATA1 1 1 87654321 CACHED MEMBER ONLINE NORMAL /dev/oracleasm/disks/DATA2

---

6. How the Mapping Works

• The cell node exposes grid disks as logical volumes.
• The compute node accesses these grid disks as ASM disks through the ASM instance.
• ASM handles the logical organization of disks into disk groups, allowing Oracle databases to store data efficiently across multiple cell nodes.

---

7. Summary of Key Mapping Components

Cell Node (Storage)	Compute Node (ASM)
Physical Disks	Not visible
Cell Disks	Not visible
Grid Disks	Appear as ASM disks
Grid Disk Name	Matches ASM disk name (DATA1)
Disk Group (asmDiskGroup)	Matches ASM disk group (DATA)
UUID/Disk ID	Matches between cell and compute

---

8. Practical Example

If DATA_CD_01 is a grid disk on the cell node:

• On the cell node:bashCopy codecellcli -e "list griddisk where name='DATA_CD_01' attributes asmDiskName,asmDiskGroup" Output:makefileCopy codeasmDiskName: DATA1 asmDiskGroup: DATA
• On the compute node:bashCopy codeasmcmd lsdsk -k Output:javascriptCopy codePath: /dev/oracleasm/disks/DATA1 Disk Group: DATA

This confirms the mapping.