Fix: Deprecation error from NumPy 1.x

[AI-Ahmed]

What’s going on?

• Problem:

The replay buffer uses np.array(..., copy=False) inside _encode_sample when building batches of actions (and possibly other fields). With NumPy 2.0, np.array(obj, copy=False) raises a ValueError whenever a copy is required, which happens when obj is a Python scalar (as is typical for env.action_space.sample()). This causes ReplayBuffer.sample(...) to crash under NumPy 2.0.

• Fix:

Replace np.array(x, copy=False) with a call that allows copying when needed, while still avoiding unnecessary copies for existing arrays. For example:

# Before (NumPy 2.0-incompatible) 

actions.append(np.array(action, copy=False))

# After (NumPy 2.0-compatible) 
actions.append(np.asarray(action))

or simply:

actions.append(np.array(action))  # omit copy=False entirely

Apply the same pattern anywhere else in the replay buffer that uses np.array(..., copy=False) with values that may be Python scalars or non-ndarray objects.

• Rationale (one sentence): This change restores compatibility with NumPy 2.0 by allowing a copy when necessary, while preserving the previous runtime behavior for NumPy 1.x (where a copy was already being made in these cases).

[dniku]

Thanks for your PR!

Looking into this, it seems .asarray() is preferred to .array(), e.g. here. Could you replace .array() with .asarray()?

Thanks!