How to Copy a Tensor in PyTorch?

PyTorch allows the user to copy the tensors using multiple methods like clone().detach(), clone(), deepcopy(), and many others. The user can also copy the tensor using the for-loop as the loop returns the values from the tensor and the user keeps storing them in another tensor. In the end, the loop has aided in creating a copy of the tensor and both of them can be used with multiple models.

Note: You can access the Google Colaboratory Python Notebook containing the code for the complete process from here

To learn the process of creating a copy of the tensor in PyTorch, simply read the following instructions:

Prerequisites

Before heading towards the main process of creating a tensor and then copying it using the PyTorch framework, make sure the following steps are completed:

Access Python Notebook
Install Modules
Import Libraries

Access Python Notebook

This guide will use the Google Colaboratory Notebook to write the Python script and it can be accessed or created by clicking on the “New Notebook” button:

Install Modules

On the notebook, install the torch framework using the pip command to get its dependencies which enables us to use its functions:

pip install torch

Import Libraries

Now, import the torch library after the framework is installed for the session:

import torch

Validate that the torch has been imported successfully by printing its installed version on the screen:

print(torch.__version__)

Creating a Tensor

Now, the torch library is available with its complete range of functions and methods to perform different tasks like creating a tensor with random values. The torch.rand() method is used to create a tensor with random numbers and the argument list explains the structure of the tensor. The single argument generates one-dimensional a tensor just like an array with 5 values as mentioned in the following code:

t = torch.rand(5)

Print the values stored in the tensor using the following code:

print(t)

Method 1: Using clone() Method

The clone() method can create shallow copies of the tensor and when called they store the tensor with the memory containing the structure of the tensors. The first method to create the copy of the tensor is using the clone() method with the name of the variable and storing it in another variable:

a = t.clone()

Print the new tensor by calling the name of the variable containing the copied tensor:

print(a)

Method 2: Using deepcopy() Method

The next method to copy the tensor is the deepcopy() method and we need to import the copy library to call it:

import copy

The deepcopy() method creates independent deep copies of the tensors and entire objects in PyTorch. It makes sure that any change in the original tensor does not affect the copied one. Create another variable to store the copied tensor and then call the deepcopy() with the copy library. Add the variable containing the original tensor as the argument while calling the method and then print the copied tensor on the screen:

tensor_copy = copy.deepcopy(t)
t

Method 3: Using new_tensor() Method

Another method called new_tensor() can be used to create a copy of the tensor by providing the name of the tensor in the argument of the function. After that, simply call the name of the variable to print the copied tensor to confirm that the copy has been created successfully:

b = t.new_tensor(t)
b

Method 4: Using clone().detach() Method

The user can also use the clone().detach() method to create the copies of the tensors in PyTorch. The method first clones the tensor with the computation path and then detaches itself from the tensor. After that, simply print the copies tensor by calling the name of the variable as mentioned in the following code:

c = t.clone().detach()
c

Note: It is not generally recommended to use the clone().detach() method as the computational path is never going to be used which makes it less efficient. To get more efficient results use the following method instead:

Method 5: Using detach().clone() Method

The next method that is “detach().clone()” function is the reverse of the previous one as it first detaches itself from the object and then clones without copying the computation path. This makes it more reasonable because copying something that will never be used is just a waste of resources. Simply use the detach().clone() method with the name of the variable of the original tensor and store it as a new variable:

f = t.detach().clone()
f

Method 6: Using empty_like().copy_() Method

Another approach for creating a copy of the tensors in PyTorch is using the empty_like() method with the name of the variable as the argument of the function. It simply creates an empty tensor with the format of the existing one and then copies the data from the tensor to create a new one:

d = torch.empty_like(t).copy_(t)
d

Method 7: Using tensor() Method

Next, the tensor() method is the most simple method of creating a tensor if the data is already available in the form of a list, or tensor. It simply creates a copy of the tensor by calling the name of the variable in the argument and storing it in another variable:

e = torch.tensor(t)
e

Method 8: Copying Data From One Tensor to Another

Now, to the last approach that explains how to copy data from one tensor to another as the structure has been set. Create a tensor with zeros stored in the tensor using the torch.zeros() method and the structure in its argument. Create another tensor with different values with the dtype argument to specify the elements of the tensor in the 8-bit unsigned integers. Create the third tensor with the values 7 and 9 in the floating data type:

x = torch.zeros(5)
y = torch.tensor(tuple((0,1,0,1,0)),dtype=torch.bool)
z= torch.tensor([7.,9.])

The following code prints the size of the tensor using its indexes as the tensor “y” only contains two non-zero elements:

print(x[y].size())

Executing the above code returns the size 2 as displayed in the screenshot below:

Now, simply assign the values stored in the third (“z”) tensor to the tensor “a” where the values of tensor “b” are non-zero:

x[y]=z

Print the values of the updated tensor after copying the values of the third tensor in the specified positions of the first one. The updated tensor contains values like 7 and 9 at the positions of 1 in the second tensor like [0, 7, 0, 9, 0]:

print(x)

Display the values stored in the second tensor in the format of a boolean datatype where 0 assumes false and 1 is replaced with true:

print(y)

That’s all about the process of copying a tensor in PyTorch using multiple methods

Conclusion

To copy a tensor in PyTorch, the framework provides multiple methods like clone(), deepcopy(), new_tensor(), clone().detach(), detach().clone(), empty_like().copy_(), and tensor(). Each of these methods works differently, however, their results are almost the same as they are used to copy the tensors using the torch framework. The user can also copy the data from one tensor to another using the indexing approach. This guide has elaborated on how to copy a tensor in PyTorch using all the above-mentioned methods.

How to Copy a Tensor in PyTorch?

Quick Outline