numpy-core

Overview

NumPy Core focuses on the foundational architecture of the ndarray. This includes how data types (dtypes) interpret raw memory, how shapes are manipulated without copying data through "views," and how memory alignment impacts hardware-level performance.

When to Use

• Creating efficient multidimensional arrays from raw data.
• Manipulating array shapes for algorithm compatibility.
• Optimizing memory-intensive operations by ensuring hardware alignment.
• Handling large-scale numerical data where precision and memory overhead are critical.

Decision Tree

1. Do you need to change array shape?
   • Use .reshape(). (Check .base to see if it's a view).
2. Are you performing truth-value testing (e.g., if arr:) on multi-element arrays?
   • No: Use .any() or .all() to avoid ambiguity errors.
3. Is performance suboptimal on modern hardware?
   • Check arr.flags['ALIGNED']. If False, use np.require with ALIGNED.

Workflows

1. Optimizing Performance via Alignment

   • Check the ALIGNED flag of the array via arr.flags.
   • Use np.require with requirements=['ALIGNED'] to ensure proper memory offsets.
   • Perform calculations on the aligned array to leverage hardware SIMD optimizations.

2. Safe Shape Manipulation

   • Use arr.reshape to create a view with a new shape without copying data.
   • Verify the result's .base attribute to confirm it is a view of the original.
   • Use .copy() if the new shape requires a contiguous layout that the view cannot provide.

3. Precision-Aware Reduction

   • Select a reduction method like .sum() or .prod().
   • Pass a larger dtype (e.g., float64 for a float32 array) to the dtype parameter.
   • Execute the operation to prevent numerical overflow during accumulation.

Non-Obvious Insights

• Shared Buffers: Different ndarrays can share the same data; modifications in a "view" are immediately visible in the base array.
• Ambiguity Prevention: NumPy raises errors for boolean testing of multi-element arrays because the truth value is ambiguous; explicit .any() or .all() is required.
• Dtype Interpretation: A dtype is not just a label but an instruction on how to interpret fixed-size blocks of memory corresponding to array items.

Evidence

• "Different ndarrays can share the same data, so that changes made in one ndarray may be visible in another." Source
• "An array is considered aligned if the memory offsets for all elements and the base offset itself is a multiple of self.itemsize." Source

Scripts

• scripts/numpy-core_tool.py: Provides utilities for checking alignment and creating precision-safe reductions.
• scripts/numpy-core_tool.js: Simulated logic for shape validation and alignment checks.

Dependencies

• numpy (Python)

References

• references/README.md