Understanding Python’s Iteration and Membership: A Information to __contains__ and __iter__ Magic Strategies

Picture by Creator

Should you’re new to Python, you might have come throughout the phrases “iteration” and “membership” and puzzled what they imply. These ideas are basic to understanding how Python handles collections of knowledge, corresponding to lists, tuples, and dictionaries. Python employs particular dunder strategies to allow these functionalities.

However what precisely are dunder strategies? Dunder/Magic strategies are particular strategies in Python that begin and finish with a double underscore, therefore the title “dunder.” They’re used to implement varied protocols and can be utilized to carry out a variety of duties, corresponding to checking membership, iterating over parts, and extra. On this article, we might be specializing in two of crucial dunder strategies: __contains__ and __iter__. So, let’s get began.

Understanding Pythonic Loops with Iter Methodology

Contemplate a fundamental implementation of a file listing utilizing Python lessons as follows:

class File:
	def __init__(self, file_path: str) -> None:
    	    self.file_path = file_path
   	 
class Listing:
	def __init__(self, information: Listing[File]) -> None:
    	    self._files = information

A simple code the place the listing has an occasion parameter that accommodates an inventory of File objects. Now, if we wish to iterate over the listing object, we should always be capable of use a for loop as follows:

listing = Listing(
	information=[File(f"file_{i}") for i in range(10)]
)
for _file in listing:
	print(_file)

We initialize a listing object with ten randomly named information and use a for loop to iterate over every merchandise. Easy sufficient, However whoops! You get an error message: TypeError: ‘Listing’ object just isn’t iterable.

What went unsuitable? ​​Effectively, our Listing class is not set as much as be looped via. In Python, for a category object to change into iterable, it should implement the __iter__ dunder methodology. All iterables in Python like Listing, Dictionaries, and Set implement this performance so we will use them in a loop.

So, to make our Listing object iterable, we have to create an iterator. Consider an iterator as a helper that offers us gadgets one after the other once we ask for them. For instance, once we loop over an inventory, the iterator object will present us with the following ingredient on every iteration till we attain the top of the loop. That’s merely how an iterator is outlined and carried out in Python.

In Python, an iterator should know the best way to present the following merchandise in a sequence. It does this utilizing a way known as __next__. When there are not any extra gadgets to offer, it raises a particular sign known as StopIteration to say, “Hey, we’re carried out right here.” Within the case of an infinite iteration, we don’t elevate the StopIteration exception.

Allow us to create an iterator class for our listing. It is going to take within the record of information as an argument and implement the following methodology to offer us the following file within the sequence. It retains observe of the present place utilizing an index. The implementation appears as follows:

class FileIterator:
    def __init__(self, information: Listing[File]) -> None:
        self.information = information
        self._index = 0
    
    def __next__(self):
        if self._index >= len(self.information):
        	elevate StopIteration
        worth = self.information[self._index]
        self._index += 1
        return worth

We initialize an index worth at 0 and settle for the information as an initialization argument. The __next__ methodology checks if the index overflows. Whether it is, it raises a StopIteration exception to sign the top of the iteration. In any other case, it returns the file on the present index and strikes to the following one by incrementing the index. This course of continues till all information have been iterated over.

Nevertheless, we aren’t carried out but! We’ve nonetheless not carried out the iter methodology. The iter methodology should return an iterator object. Now that we have now carried out the FileIterator class, we will lastly transfer in direction of the iter methodology.

class Listing:
    def __init__(self, information: Listing[File]) -> None:
        self._files = information
    
    def __iter__(self):
        return FileIterator(self._files)

The iter methodology merely initializes a FileIterator object with its record of information and returns the iterator object. That is all it takes! With this implementation, we will now loop over our Listing construction utilizing Python’s loops. Let’s examine it in motion:

listing = Listing(
	information=[File(f"file_{i}") for i in range(10)]
)
for _file in listing:
	print(_file, finish=", ")

# Output: file_0, file_1, file_2, file_3, file_4, file_5, file_6, file_7, file_8, file_9,

The for loop internally calls the __iter__ methodology to show this end result. Though this works, you would possibly nonetheless be confused in regards to the underlying workings of the iterator in Python. To grasp it higher, let’s use some time loop to implement the identical mechanism manually.

listing = Listing(
	information=[File(f"file_{i}") for i in range(10)]
)

iterator = iter(listing)
whereas True:
    strive:
        # Get the following merchandise if accessible. Will elevate StopIteration error if no merchandise is left.
        merchandise = subsequent(iterator)   
        print(merchandise, finish=', ')
    besides StopIteration as e:
        break   # Catch error and exit the whereas loop

# Output: file_0, file_1, file_2, file_3, file_4, file_5, file_6, file_7, file_8, file_9,

We invoke the iter perform on the listing object to accumulate the FileIterator. Then, we manually make the most of the following operator to invoke the following dunder methodology on the FileIterator object. We deal with the StopIteration exception to gracefully terminate the whereas loop as soon as all gadgets have been exhausted. As anticipated, we obtained the identical output as earlier than!

Testing for Membership with Incorporates Methodology

It’s a pretty widespread use case to test for the existence of an merchandise in a set of objects. For instance in our above instance, we might want to test if a file exists in a listing very often. So Python makes it easier syntactically utilizing the “in” operator.

print(0 in [1,2,3,4,5]) # False
print(1 in [1,2,3,4,5]) # True

These are majorly used with conditional expressions and evaluations. However what occurs if we do this with our listing instance?

print("file_1" in listing)  # False
print("file_12" in listing) # False

Each give us False, which is inaccurate! Why? To test for membership, we wish to implement the __contains__ dunder methodology. When it’s not carried out, Python fall backs to utilizing the __iter__ methodology and evaluates every merchandise with the == operator. In our case, it’ll iterate over every merchandise and test if the “file_1” string matches any File object within the record. Since we’re evaluating a string to customized File objects, not one of the objects match, leading to a False analysis

To repair this, we have to implement the __contains__ dunder methodology in our Listing class.

class Listing:
    def __init__(self, information: Listing[File]) -> None:
        self._files = information
    
    def __iter__(self):
        return FileIterator(self._files)
    
    def __contains__(self, merchandise):
        for _file in self._files:
        	# Examine if file_path matches the merchandise being checked
        	if merchandise == _file.file_path:
            	return True
    	return False

Right here, we modify the performance to iterate over every object and match the file_path from the File object with the string being handed to the perform. Now if we run the identical code to test for existence, we get the right output!

listing = Listing(
	information=[File(f"file_{i}") for i in range(10)]
)

print("file_1" in listing)	# True
print("file_12" in listing) # False

Wrapping Up

And that’s it! Utilizing our easy listing construction instance, we constructed a easy iterator and membership checker to grasp the interior workings of the Pythonic loops. We see such design selections and implementations pretty usually in production-level code and utilizing this real-world instance, we went over the integral ideas behind the __iter__ and __contains__ strategies. Maintain working towards with these methods to strengthen your understanding and change into a more adept Python programmer!
 
 

Kanwal Mehreen Kanwal is a machine studying engineer and a technical author with a profound ardour for knowledge science and the intersection of AI with medication. She co-authored the book “Maximizing Productiveness with ChatGPT”. As a Google Technology Scholar 2022 for APAC, she champions variety and educational excellence. She’s additionally acknowledged as a Teradata Range in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower girls in STEM fields.