As you know, the available capacity of Google Drive is limited, and you can not upload big datasets to your Drive.

I will show you a method to use the shared folder of other people in your code.

For example, I want to use the WIDER Face dataset for face detection.

It supports Google Drive. So, click on “Add shortcut to Drive”.

Then select the folder that you want to use this dataset.

After a successful operation, you will see the following result :

The shows that it is a shortcut.

This lesson is the final part of the function study in Python, and the remaining topics from the Python function are discussed in this lesson.

✔ Level: Medium

Headlines

Lesson 14: Function in Python: Recursive Function and Memoization

recursive function
Function Attributes
Built-in Functions
Documentation Strings

recursive function

From Lesson 9 we are introduced to the for and while control commands, which were our only tools for repeating part of the code. We are now introduced to the implementation of new methods of repetition.

Simply put, a recursive function is a function that calls itself from inside its body. Implementing a recursive function is a method used to solve some problems, and we should know that recursive functions are not a specific syntax or command in Python, but a problem-solving method that uses a function in the Python programming language (e.g. Many other languages) can be implemented.

For example, in the example code below, we calculate the factorial value of five recursively:

>>> def factorial(n):
...     if n <= 1:
...         return 1
...     else:
...         return n * factorial(n - 1)
...
>>>

>>> factorial(5)
120
>>>

Generally, problems that can be solved from the sequence of doing the same task can be implemented recursively. The steps to execute the sample code above are as follows:

factorial(5)
|--> 5 * factorial(4)
         |--> 4 * factorial(3)
                  |--> 3 * factorial(2)
                           |--> 2 * factorial(1)
                                    |--> 1

120 = 5 * (4 * (3 * (2 * 1)))

Explanation: When factorial (5) is called (n == 5), the condition 1 => n is rejected and the else section is executed. In this step, another instance of the function with argument 4 is called and the factorial (5) execution waits for the end of the factorial (4) execution and receives the result. Similarly, several instances of a function are executed, waiting to receive the result from the next instance. Finally, the condition 1 => n is established and the factorial (1) instance returns its result to factorial (2). In the same way, the results are returned to reach the first executed instance, ie factorial (5), and the execution desired by the user is completed.

Sequence management of a function (recursive method) in memory is done using the Stack data structure [Wikipedia].
Each recursive function consists of two important parts:

• A phrase containing a self-calling function
• A condition for choosing between redial and end

Attention

Implementing a recursive method may seem exciting, but it should not be forgotten that it consumes a lot of memory, it will take time, it is often difficult to understand the execution process, and debugging will not be easy!

Use a decorator

When using decorator on recursive functions, note that this decorator will be applied to all called instances of the function, and that only one instance of the decorator is created, and all instance instances of the function are sent to the same instance:

>>> def logger(func):
...     print('Decorator is created!')
...     def func_wrapper(number):
...         print(f'New factorial call with parameter: 
{number}')
...         result = func(number)
...         print (f'factorial({number}) ==> {result}')
...         return result
...     return func_wrapper
...

>>> @logger
... def factorial(n):
...     if n <= 1:
...         return 1
...     else:
...         return n * factorial(n - 1)
...
>>>

>>> factorial(5)
Decorator is created!
New factorial call with parameter: 5
New factorial call with parameter: 4
New factorial call with parameter: 3
New factorial call with parameter: 2
New factorial call with parameter: 1
factorial(1) ==> 1
factorial(2) ==> 2
factorial(3) ==> 6
factorial(4) ==> 24
factorial(5) ==> 120
120
>>>

Be sure to pay attention to the sample output of the above code !.

Adjusting the recursive depth

In the Python programming language, there is an adjustable limit to the depth of implementation of recursive functions (the number of instances called from the function and in the stack). The getrecursionlimit () function of the sys module returns this value [Python Documents]. This value is 1000 by default, which can be changed using the setrecursionlimit (limit) function of the sys module [Python Documents]:

>>> import sys

>>> sys.getrecursionlimit()
1000

>>> sys.setrecursionlimit(50)

>>> sys.getrecursionlimit()
50

A RecursionError exception will be reported by exceeding the depth limit of the return functions:

>>> factorial(9)
362880

>>> sys.setrecursionlimit(10)

>>> factorial(9)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in factorial
  File "<stdin>", line 5, in factorial
  File "<stdin>", line 5, in factorial
  [Previous line repeated 5 more times]
  File "<stdin>", line 2, in factorial
RecursionError: maximum recursion depth exceeded in 
comparison

tip

In addition to this limitation, there is another more serious limitation, and that is the amount of space provided by the operating system for the stack. By passing this amount of space, the program encounters a runtime error (RuntimeError).

Recursive Generator function

Implementing the Generator and Coroutine functions can also be considered a recursive method, in which case the results may be slightly contrary to your expectations. The following code sample receives a nested list object and prints each member in each list:

>>> def flatten(lists):
...     for sub in lists:
...         if isinstance(sub,list):
...             flatten(sub)
...         else:
...             print(sub)
...

>>> items = [[1,2,3],[4,5,[5,6]],[7,8,9]]

>>> flatten(items)
1
2
3
4
5
5
6
7
8
9
>>>

Now to convert the flatten function to a Generator, it is enough to use yield instead of print:

>>> def genflatten(lists):
...     for sub in lists:
...         if isinstance(sub,list):
...             genflatten(sub)
...         else:
...             yield sub
...

>>> items = [[1,2,3],[4,5,[5,6]],[7,8,9]]

>>> genflatten(items)
<generator object genflatten at 0x7eff06d40150>

>>> list(genflatten(items))
[]

Nothing happened! And the output is an empty list. Remember from the previous lesson, calling the genflatten function (which is actually a Generator function) only creates a Generator object, and we must also prepare for the output processing of a Generator object at the point where the function calls itself. Now with the above code modification:

>>> def genflatten(lists):
...     for sub in lists:
...         if isinstance(sub,list):
...             for item in genflatten(sub):
...                 yield item
...         else:
...             yield sub
...

>>> items = [[1,2,3],[4,5,[5,6]],[7,8,9]]

>>> genflatten(items)
<generator object genflatten at 0x7f6cee349258>

>>> list(genflatten(items))
[1, 2, 3, 4, 5, 5, 6, 7, 8, 9]

Memoization

Memoization is a technique for storing results to prevent duplication of calculations [Wikipedia]. This technique can be implemented in the Python programming language using a decorator.

To illustrate this point, let’s look at another recursive example. Calculate the Fibonacci value [Wikipedia] of a specific number:

>>> def fibonacci(n):
...     if n <= 1:
...         return n
...     else:
...         return fibonacci(n-1) + fibonacci(n-2)
...

>>> for number in range(10):
...    print(fibonacci(number))
...
0
1
1
2
3
5
8
13
21
34

In this example we did not go beyond the number 9 because the calculation for larger numbers like 50 will be really time consuming and this is an opportunity for us to test the efficiency of the Memoization technique. Now we optimize our Fibonacci return function using the Memoization technique and a Decorator:

>>> def memoize_fibonacci(func):
...     memory = {}
...     def func_wrapper(number):
...         if number not in memory:
...             memory[number] = func(number)
...         return memory[number]
...     return func_wrapper
...

>>> @memoize_fibonacci
... def fibonacci(n):
...     if n <= 1:
...         return n
...     else:
...         return fibonacci(n-1) + fibonacci(n-2)
...
>>>

Now calculate the value of 50 if not, the Fibonacci value for the number 500 ((500) Fibonacci). You will see the difference in runtime yourself!

 With the help of Decorator in this example (memoize_fibonacci), the results of calling each instance of the function are stored somewhere (memory dictionary object) and before retrying a new instance of the function, it is checked whether this value has already been calculated or not. If there is an answer, the function is ignored again and the pre-existing value is returned as a result. Therefore, it is obvious that the execution speed will increase by preventing the creation of samples of new functions and duplicate calculations.

Function Attributes

We have seen from previous lessons that objects in Python contain a set of default attributes depending on their type; For example, the __name__ attribute, which contains the function name [Python Documents].

In addition, functions in Python can also receive user-desired attributes, so that a series of additional information can be attached to the functions [PEP 232]. Note the example code below:

>>> def foo():
...     pass
...

>>> foo.is_done = True
>>>

>>> if foo.is_done:
...     print('DONE!')
...
DONE!
>>>

As can be seen, an Attribute can be added to the function using the following syntax:

function_name.attribute_name = attribute_value

You can also use the ready-made function setattr (object, name, value [Python Documents]) for this purpose. This function receives three arguments: the object to which you want to add an attribute (here the function), the name (of the string type) – string) and the desired Attribute value:

>>> setattr(foo, 'name', 'Saeid')

>>> setattr(foo, 'age', 32)
>>>

>>> foo.name
'Saeid'

>>> foo.age
32

These attributes are stored in a dictionary object that are accessible using the __dict__ attribute [Python Documents]:

>>> foo.__dict__
{'is_done': True, 'name': 'Saeid', 'age': 32}

To get the value of a specific attribute you can also use the ready-made function getattr object, name [, default] [Python Documents]. This function has two mandatory parameters (object and name) and an optional parameter (default). The object (function here) does not have the desired attribute, the default value (if sent) will be returned:

>>> getattr(foo, 'is_done')
True

>>> getattr(foo, 'is_publish', False)
False

>>> getattr(foo, 'is_publish')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 
'is_publish'

>>> foo.is_publish
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 
'is_publish'

An AttributeError exception will be reported if you try to get an attribute that is not defined for the function. Of course, as mentioned, this will not happen if you use the getattr function and set the default parameter. To prevent this exception, you can also check the existence of an attribute using the hasattr (object, name) function before using it [Python Documents]:

>>> if hasattr(foo, 'is_publish'):
...     print(foo.is_publish)
... else:
...     print(f"{foo.__name__!r} has no attribute 
'is_publish'")
...
'foo' has no attribute 'is_publish'
>>>

You can also use the delattr (object, name) function to delete an attribute [Python Documents]:

>>> delattr(foo, 'age')
>>>

>>> foo.age
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 'age'

Or from the del command

>>> del foo.is_done
>>>

>>> foo.is_done
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 
'is_done'
>>>

Attention

At the end of this section, it should be noted that when defining an attribute for its functions and using a decorator, as described in the previous lesson, do not forget to use @ functools.wraps [Lesson 13].

Built-in Functions

The Python interpreter provides programmers with a number of functional functions without the need to import a specific module. These functions are called Built-in Functions. A complete list of these functions is available in the Python documentation with descriptions. You have become familiar with some of them during the previous lessons and even this lesson, in this section we will examine some other cases.

eval

This function returns one (and only one) Python expression in the form of a string object to receive, execute, and return [Python documents].

>>> eval('3*4 + 7.2')
19.2

>>> import math

>>> x = 2

>>> eval('math.sin(3.5+x) + 7.2')
6.494459674429608

According to the definition in the Python documents eval object [, globals], local, this function also includes two parameters, globals and locals, to which arguments are optional. Both are of the dictionary type, which is Scope or The global and local domains provide the code to be executed (function parameter 1):

>>> globals_env = {'x': 7, 'y': 10, 'birds': ['Parrot', 
'Swallow', 'Albatross']}

>>> locals_env = {}

>>> a = eval("3 * x + 4 * y", globals_env, locals_env)

>>> a
61

exec

This function is the same as eval, except that it can receive and execute multiple Python statements or commands in a single string object. Exec output is always None [Python documents].

>>> exec('import math; x=2; print
(math.sin(3.5+x) + 7.2)')
6.494459674429608

>>> exec("for i in range(5): print(i)")
0
1
2
3
4

Attention

exec in Python version 2x is not defined as a function and is used as a command [Python Documents]:

>>> exec 'import math; x=2; print
(math.sin(3.5+x) + 7.2)'
6.49445967443

This function, like eval, includes two parameters, globals and locals:

exec(object[, globals[, locals]])

>>> exec("for b in birds: print(b)", 
globals_env, locals_env)
Parrot
Swallow
Albatross

Of course, in 2x versions, the syntax exec code [in globals], locals is followed:

>>> exec "for b in birds: print b" in globals_env, 
locals_env
Parrot
Swallow
Albatross

compile

Each time a string object containing Python code is passed to the eval and exec functions, the Python interpreter first compiles the code into the bytecode and then executes it, repeating this overload. This overhead can be avoided once the computer is reused.

The compile function is for this purpose [Python Documents]. The definition of this function is as follows:

compile(source, filename, mode, flags=0, 
dont_inherit=False, 
optimize=-1)

• source: is the code that we want to computerize and eventually execute, which can be an object of type str (str), bytes (bytes) or AST [Python documents].
• filename: The name of the file from which the code should be read; If the code you want is not read from the file, put a name of your choice or set it to a blank string.
• mode: Specifies the type of code. It can be one of the exec, eval or single values. We examined the execution conditions of the two functions eval (containing only one expression) and exec (one or more expressions and commands), and single is also used when the code in question contains only one command.
• flags, dont_inheritmode: These two parameters are optional and you can skip them at this point. The two are used to determine which of the Future statements is effective in compiling the code [Python Documents].
• optimize: Adjusts the level of code optimization for the compiler, and sending arrogant to it is also optional – Further reading: [PEP 488].

Note the sample codes below:

>>> # compile() with string source

>>> code_str = 'x=5\ny=10\nprint("sum =",x+y)'

>>> code = compile(code_str, 'sum_test.py', 'exec')

>>> print(type(code))
<class 'code'>

>>> exec(code)
sum = 15

1
2
3
4
5
6

# File Name: test_code.py
# Directory: /home/saeid/Desktop

x = 10
y = 20
print('Multiplication = ', x * y)

>>> # reading code from a file

>>> file = open('/home/saeid/Desktop/test_code.py', 'r')

>>> code_str = file.read()

>>> file.close()

>>> code = compile(code_str, 'test_code.py', 'exec')

>>> print(type(code))
<class 'code'>

>>> exec(code)
Multiplication =  200

locals, globals

The output of both functions is a dictionary object (dict). The locals () function returns a dictionary containing variables in the local [Python documents] domain, and the globals () function returns a dictionary containing variables in the global [Python documents]:

>>> a = 0

>>> def func():
...     x = 'text'
...     print('-' * 10)
...     print('locals():')
...     print(locals())
...     print('-' * 10)
...     print('globals():')
...     print(globals())
...

>>> func()
----------
locals():
{'x': 'text'}
----------
globals():
{'__name__': '__main__', '__doc__': None, 
'__package__': 
None, '__loader__'
: <class '_frozen_importlib.BuiltinImporter'>, 
'__spec__': 
None, '__annotations__': {}, '__builtins__': 
<module 'builtins' (built-in)>, 'a': 0, 'func': 
<function func at 0x7f2b29f1ec80>}
>>>

Note that at the output module level these two functions are identical:

>>> a = 5

>>> b = 10

>>> def func():
...     pass
...

>>> locals()
{'__name__': '__main__', '__doc__': None, '__package__': 
None, '__loader__': 
<class '_frozen_importlib.BuiltinImporter'>, 
'__spec__': None, '__annotations__': {}, '__builtins__': 
<module 'builtins' (built-in)>, 'a': 5, 'b': 10, 'func': 
<function func at 0x7f1dd0218c80>}

>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': 
None, '__loader__': 
<class '_frozen_importlib.BuiltinImporter'>, '__spec__': 
None, '__annotations__': {}, '__builtins__': 
<module 'builtins' (built-in)>, 'a': 5, 'b': 10, 'func': 
<function func at 0x7f1dd0218c80>}
>>>

consideration

As stated in Lessons 3 and 4, the entire Python interactive environment is like a module (or script) to the Python interpreter.

Documentation Strings

From the sixth lesson we are familiar with docstring; In this section, we address this issue with a function approach [PEP 257].

tip

Using docstring at the beginning of modules, classes, and functions is a good way in Python to show how these elements relate to and behave.

Note the example code below:

>>> def factorial(n):
...     """Computes n factorial. For example:
...
...     >>> factorial(5)
...     120
...     >>>
...     """
...
...     if n <= 1: return 1
...     else: return n*factorial(n-1)
...
>>>

>>> factorial.__doc__
'Computes n factorial. For example:\n\n 
   
>>> factorial(5)\n    120\n    >>>\n    '

>>> print(factorial.__doc__)
Computes n factorial. For example:

    >>> factorial(5)
    120
    >>>
>>>

>>> help(factorial)

Help on function factorial in module __main__:

factorial(n)
    Computes n factorial. For example:

    >>> factorial(5)
    120
    >>>
(END)

The value of docstring is placed in the attribute __doc__ of the function. This value is also available through the help function in the Python interactive environment.

IDE programs, such as PyCharm, also process docstrings and use the information in them to provide more help to the programmer. For example, the input type of a function or its output value can be described using docstring. For more information and to view sample code you can refer to the PyCharm documentation: PyCharm – Document source code

😊 I hope it was useful

This lesson is a continuation of the previous lesson, which introduces some of the applications of the function in creating new, important and practical concepts in the Python programming language. The topic of function in Python does not end with this lesson, and the remaining points are presented in the next lesson.

✔ Level: Medium

Headlines

Lesson 13: Function in Python: Coroutine, Generator, Decorator‌ and Lambda

Decorator
Generator
Continued Coroutine: yield
List Comprehensions
Generator Expressions
lambda and anonymous functions

Decorator

Decorators [PEP 318] are functions that are implemented to wrap other functions or classes. Decorators in Python are very useful tools that allow the programmer to expand their behavior and features by reducing the coding volume without changing the body of their functions and classes. In this section, the focus is on applying Decorators to functions, and we will examine Class Decorator in a class lesson.

To decorate a function by Decorator‌, a syntax similar to decorator‌_name @ is used at the top of the header:

def decorator_name(a_function):
    pass


@decorator_name
def function_name():
    print("Somthing!")


function_name()

The concept that this syntax (decorator‌_name + @) at the top of the header creates a function for the Python interpreter is quite similar to the syntax below:

wrapper = decorator_name(function_name)
wrapper()

Everything in Python is an object, even complex concepts such as a function; We also remember from the previous lesson that a function in Python is a “first-class” entity, which means that a function can be passed as an argument to other functions like other objects. An example of the above code is the display of sending a function (function_name) to another function (decorator‌_name).

Consider the example below:

>>> def decorator_name(func):
...     def wrapper():
...         print("Something is happening before 
the function is called.")
...         func()
...         print("Something is happening after 
the function is called.")
...     return wrapper
...
>>>

>>> @decorator_name
... def function_name():
...     print("Somthing!")
...
>>>

>>> function_name()
Something is happening before the function is called.
Somthing!
Something is happening after the function is called.
>>>

The above code example can also be considered with the following simple structure:

>>> def decorator_name(func):
...     def wrapper():
...         print("Something is happening before 
the function is called.")
...         func()
...         print("Something is happening after 
the function is called.")
...     return wrapper
...
>>>

>>> def function_name():
...     print("Somthing!")
...
>>>

>>> wrapper = decorator_name(function_name)

>>> wrapper()
Something is happening before the function is called.
Somthing!
Something is happening after the function is called.
>>>

As can be seen by comparing the two examples of code above, Decorators create a wrapper for our functions and classes. When calling the function_name function, the Python interpreter notices its decorator, and instead of executing it, it sends an instance of the object to the specified decorator (decorator‌_name) and receives and executes a new object specified here with the wrapper function.

In the case of parametric functions, it should also be noted that when calling the desired function and sending the argument to the function, the Python interpreter sends these arguments to the wrapper function of decorator:

>>> def multiply_in_2(func):
...     def wrapper(*args):
...         return func(*args) * 2
...     return wrapper
...
>>>

>>> @multiply_in_2
... def sum_two_numbers(a, b):
...     return a + b
...
>>>

>>> sum_two_numbers(2, 3)
10

>>> # normal
>>>

>>> def multiply_in_2(func):
...     def wrapper(*args):
...         return func(*args) * 2
...     return wrapper
...
>>>

>>> def sum_two_numbers(a, b):
...     return a + b
...
>>>

>>> wrapper = multiply_in_2(sum_two_numbers)

>>> wrapper(2, 3)
10

More than one Decorator can be applied to its classes and functions, in which case the order in which these Decorators are arranged is important to the Python interpreter:

@decorator_3
@decorator_2
@decorator_1
def function_name():
    print("Somthing!")


function_name()

wrapper = decorator_3
(decorator_2(decorator_1(function_name)))
wrapper()

Arguments can also be sent to Decorators‌:

@decorator_name(params)
def function_name():
    print("Somthing!")


function_name()

In this case, the Python interpreter first sends the argument to the Decorator‌ function and then calls the result with the input argument of the desired function:

temp_decorator = decorator_name(params)
wrapper = temp_decorator(function_name)
wrapper()

Note the example code below:

>>> def formatting(lowerscase=False):
...     def formatting_decorator(func):
...         def wrapper(text=''):
...             if lowerscase:
...                 func(text.lower())
...             else:
...                 func(text.upper())
...         return wrapper
...     return formatting_decorator
...
>>>

>>> @formatting(lowerscase=True)
... def chaap(message):
...     print(message)
...
>>>

>>> chaap("I Love Python")
i love python
>>>

functools.wraps @

In Python, there is a term called Higher-order functions, which refers to functions that perform operations on other functions or return a new function as output. Accordingly, a module called functools is located in the standard Python library, which provides a series of auxiliary and functional functions for such functions [Python documents]. One of the functions inside this module is wraps.

But why is it important to introduce this function in this section? When we use a Decorator‌, what happens is that a new function replaces our main function. Note the sample codes below:

>>> def func(x):
...     """does some math"""
...     return x + x * x
...
>>>

>>> print(func.__name__)
func

>>> print(func.__doc__)
does some math
>>>

>>> def logged(func):
...     def with_logging(*args, **kwargs):
...         print(func.__name__ + " was called")
...         return func(*args, **kwargs)
...     return with_logging
...
>>>

>>> @logged
... def f(x):
...     """does some math"""
...     return x + x * x
...
>>>

>>> print(f.__name__)
with_logging

>>> print(f.__doc__)
None
>>>

>>> # It is mean: f = logged(func)
...

>>> f = logged(func)

>>> print(f.__name__)
with_logging

When using Decorator‌, when we wanted to print the function name (__print (f .__ name), the name of the new function (with_logging) was printed, not the original function (f).

Using Decorator‌ always means losing the information about the main function, so we can use the wraps function to prevent this from happening and to preserve the information about our main function. This function is itself a Decorator‌ whose job it is to copy information from the function it receives as an argument to the function to which it is assigned:

>>> from functools import wraps
>>>

>>> def logged(func):
...     @wraps(func)
...     def with_logging(*args, **kwargs):
...         print(func.__name__ + " was called")
...         return func(*args, **kwargs)
...     return with_logging
...
>>>

>>> @logged
... def f(x):
...    """does some math"""
...    return x + x * x
...
>>>

>>> print(f.__name__)
f

>>> print(f.__doc__)
does some math
>>>

Please also note the last example of the Decorator‌ discussion. In this example we will calculate the execution time of a function using Decorators‌ [Source]:

>>> import functools

>>> import time
>>>

>>> def timer(func):
...     """Print the runtime of the decorated function"""
...     @functools.wraps(func)
...     def wrapper_timer(*args, **kwargs):
...         start_time = time.perf_counter()
...         value = func(*args, **kwargs)
...         end_time = time.perf_counter()
...         run_time = end_time - start_time
...         print(f"Finished {func.__name__!r} in 
{run_time:.4f} secs")
...         return value
...     return wrapper_timer
...
>>>

>>> @timer
... def waste_some_time(num_times):
...     result = 0
...     for _ in range(num_times):
...         for i in range(10000)
...             result += i**2
...
>>>

>>> waste_some_time(1)
Finished 'waste_some_time' in 0.0072 secs

>>> waste_some_time(999)
Finished 'waste_some_time' in 2.6838 secs

In this example, the perf_counter function is used to calculate time intervals, which is only available from version 3.3 onwards.

If you do not understand the print command code in the wrapper_timer function, refer to lesson 7 of the f-string section [lesson 7 f-string].

Generator

Generators [PEP 255] are functions that are implemented to create a function with behavior similar to iterator objects (iterator – lesson 9).

When a normal function is called, the body of the function is executed to reach a return statement and ends, but by calling a Generator‌ function, the body of the function is not executed but a generator object is returned, which can be done using the __ (next__) method () or () next In Python 2x), it requested its expected values ​​one after the other.

Generator‌ function is lazy [Wikipedia] and does not store data together but only generates it when requested. So when dealing with large data sets, Generators have more efficient memory management, and we also do not have to wait for a sequence to be generated before using it all!

To create a Generator function, it is enough to use one or more yield commands in a normal function. The Python interpreter now returns a generator object when calling such a function, which has the ability to generate a sequence of values ​​(or objects) for use in repetitive applications.

The syntax of the yield statement is similar to the return statement, but with a different application. This command stops the execution of the program at any point in the body of the function, and we can use the __ next__ () method (or (next) in Python 2x) to get the yield value:

>>> def a_generator_function():
...    for i in range(3):  # i: 0, 1, 2
...       yield i*i
...    return
...

>>> my_generator = a_generator_function()  
# Create a generator
>>>

>>> my_generator.__next__()  #  Use my_generator.next() 
in Python 2.x
0

>>> my_generator.__next__()
1

>>> my_generator.__next__()
4

>>> my_generator.__next__()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

It should be noted that the end of the Generator function generation process is reported by the StopIteration exception. However, when using commands such as for, this exception is controlled and the loop ends. Rewrite the previous code example as follows:

>>> def a_generator_function():
...    for i in range(3):  # i: 0, 1, 2
...       yield i*i
...    return
...
>>>

>>> for i in a_generator_function():
...     print(i)
...
0
1
4
>>>

To better understand the function of the Generator‌ function, imagine that you are asked to implement a personal function similar to the Python range () function. What will be your solution? Create an object like a list or an empty tuple and fill it with a loop ?! This solution may be responsible for creating small intervals, but do you have enough memory and time to create a 100 million range? We will solve this problem easily and correctly using the Generator‌ function:

>>> def my_range(stop):
...     number = 0
...     while number < stop:
...         yield number
...         number = number + 1
...     return
...
>>>

>>> for number in my_range(100000000):
...     print(number)

Features of the Generator‌ function

• The Generator‌ function contains one or more yield statements.
• When the Generator‌ function is called, the function is not executed, but instead an object of generator type is returned for that function.
• Using the yield command, we can pause anywhere in the Generator ‌ function and get the yield obtained using the __ next__ () or (next) (in Python 2x) method. The first call to the (__ next__) method executes the function until it reaches a yield statement. At this time the yield statement produces a result and the execution of the function stops. By re-calling the __ (next__) method, the execution of the function resumes from the continuation of the same yield statement.
• There is usually no need to use the __ next__ () method directly, and Generator‌ functions are used through commands such as for or functions such as sum (etc.) that have the ability to receive a sequence.
• At the end of generating the Generator‌ functions, they report a StopIteration exception at their stopping point that must be controlled within the program.
• Let’s not forget that using the return command anywhere in the body of the function ends the execution of the function at that point, and the Generator‌ functions are no exception!
• You can turn off a Generator‌ object by calling the close method! Note that a StopIteration exception is reported after this method is called again if the send value ((__ next__)) is requested.

Consider another example code:

>>> def countdown(n):
...     print("Counting down from %d" % n)
...     while n > 0:
...        yield n
...        n -= 1
...     return
...
>>>

>>> countdown_generator = countdown(10)
>>>

>>> countdown_generator.__next__()
Counting down from 10
10

>>> countdown_generator.__next__()
9

>>> countdown_generator.__next__()
8

>>> countdown_generator.__next__()
7
>>>

>>> countdown_generator.close()
>>>

>>> countdown_generator.__next__()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

tip

The Generator object can be converted to a list object using the list () function:

>>> countdown_list = list(countdown(10))
Counting down from 10
>>>

>>> countdown_list
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
>>>

Continued Coroutine: yield

New features have been added to the Generator‌ function since Python 2.5 [PEP 342]. If inside a function, we put the yield statement to the right of an assignment = operator, then that function exhibits a different behavior, which in the Python programming language is called Coroutine. Imagine now we can send our desired values ​​to the Generator‌ function !:

>>> def receiver():
...     print("Ready to receive")
...     while True:
...         n = (yield)
...         print("Got %s" % n)
...
>>>


>>> receiver_generator = receiver()

>>> receiver_generator.__next__() # python 3.x - 
In Python 2.x use .next()
Ready to receive

>>> receiver_generator.send('WooW!!')
Got WooW!!

>>> receiver_generator.send(1)
Got 1

>>> receiver_generator.send(':)')
Got :)

How to run a Coroutine is the same as a Generator‌, except that the send () method is also available to send the value into the function.

Calling the Coroutine function does not execute the body, but returns a Generator. Object. The __ next__ () method (or (next) in Python 2x) brings the execution of the program to the first yield, at which point the function is suspended and ready to receive the value. The send () method sends the desired value to the function, which is received by the expression (yield) in Coroutine. After receiving the value, the execution of Coroutine continues until the next yield (if any) or the end of the body of the function.

In the Coroutine discussion, Decorators can be used to get rid of the __ next__ () method call:

>>> def coroutine(func):
...     def start(*args,**kwargs):
...         generator = func(*args,**kwargs)
...         generator.__next__()
...         return generator
...     return start
...
>>>

>>> @coroutine
... def receiver():
...     print("Ready to receive")
...     while True:
...         n = (yield)
...         print("Got %s" % n)
...
>>>

>>> receiver_generator = receiver()

>>> receiver_generator.send('Hello World')  
# Note : No initial .next()/.__next__() needed

A coroutine can be executed indefinitely unless it is executed by the program by calling the close () method or by itself ending the execution lines of the function.

If the send () method is called after the Coroutine expires, a StopIteration exception will occur:

>>> receiver_generator.close()

>>> receiver_generator.send('value')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

A Coroutine can generate and return output at the same time as receiving the value:

>>> def line_splitter(delimiter=None):
...     print("Ready to split")
...     result = None
...     while True:
...         line = yield result
...         result = line.split(delimiter)
...
>>>

>>> splitter = line_splitter(",")
>>>

>>> splitter.__next__()  # python 3.x - 
In Python 2.x use .next()
Ready to split
>>>

>>> splitter.send("A,B,C")
['A', 'B', 'C']
>>>

>>> splitter.send("100,200,300")
['100', '200', '300']
>>>

What happened?!

 The line_splitter function is called with the input value “,”. As we know, at this point, the only thing that happens is to create a generator‌ object instance (and none of the lines inside the function body will be executed). By calling the method () __ splitter .__ next the body of the function is executed to reach the first yield. That is, the phrase “Ready to split” in the print output defines the result variable with the initial value None and finally reaches the line = yield result line by confirming the condition of the command while executing. In this row, based on the evaluation of the expression to the right of the assignment operation, the value of the result variable equal to None is returned outside the function and then the function is suspended. However, it should be noted that the assignment in this line has not been completed yet! Then, by calling the method “splitter.send” (“A, B, C”, the string “A, B, C” is placed in the yield and the execution of the program is suspended and continues. The yield value is assigned to the line and The execution of the line = yield result is complete.In the next line, the string inside the line variable is separated based on the delimiter, which was initially set to “,” and assigned to the result variable (the value of the result variable, which was previously None) With the end of the body lines and the confirmation of the while statement condition again, the body is executed again to reach the yield again, ie to the line line = yield result. Now in the second run of the loop, unlike the first time, the value of the result variable Is not equal to None and its yield or return operation will be visible in the output, ie the value [‘A’, ‘B’, ‘C’] that was generated the first time the loop was executed, will now be displayed in the output Comes and then the function is suspended again (the function waits for one of the methods to send () send or (__ next__) or () close.) The procedure for calling the method (“splitter.send” (100,200, 300 continues like this …

In the case of the line = yield result line, we know that in order to perform the assignment operation, it is necessary to first evaluate the value of the expression on the right and then assign it to the left. That is, the Python interpreter first executes the yield result, the result of which is to return the value of the result variable (in the first run of the loop = None) to the outside of the function, and then the line = yield statement, which assigns the value of the send () method to the line variable.

The topic of Coroutine is broader than the level that can be covered in this lesson, but at this point, David Beazley’s presentation at PyCon’2009 can be useful for more examples, applications, and details on the topic of Coroutine, the Python programming language.

PDF: [A Curious Course on Coroutines and Concurrency]

VIDEO: [YouTube]

List Comprehensions

List Comprehensions is an operation during which a function can be applied to each member of a list object type and the result can be obtained in the form of a new list object type [PEP 202]:

>>> numbers = [1, 2, 3, 4, 5]

>>> squares = [n * n for n in numbers]
>>>

>>> squares
[1, 4, 9, 16, 25]
>>>

The above code sample is equal to:

>>> numbers = [1, 2, 3, 4, 5]

>>> squares = []

>>> for n in numbers:
...     squares.append(n * n)
...
>>>

>>> squares
[1, 4, 9, 16, 25]

The general syntax of List Comprehensions is as follows:

[expression for item1 in iterable1 if condition1
    for item2 in iterable2 if condition2
    ...
    for itemN in iterableN if conditionN]

# This syntax is roughly equivalent to 
the following code:

s = []
for item1 in iterable1:
    if condition1:
        for item2 in iterable2:
            if condition2:
            ...
               for itemN in iterableN:
                   if conditionN: s.append(expression)

Consider other examples in this regard:

>>> a = [-3,5,2,-10,7,8]

>>> b = 'abc'

>>> [2*s for s in a]
[-6, 10, 4, -20, 14, 16]

>>> [s for s in a if s >= 0]
[5, 2, 7, 8]

>>> [(x,y) for x in a for y in b if x > 0]
[(5, 'a'), (5, 'b'), (5, 'c'), (2, 'a'), (2, 'b'), 
(2, 'c'), 
(7, 'a'), (7, 'b'), (7, 'c'), (8, 'a'), (8, 'b'), 
(8, 'c')]

>>> import math

>>> c = [(1,2), (3,4), (5,6)]

>>> [math.sqrt(x*x+y*y) for x,y in c]
[2.23606797749979, 5.0, 7.810249675906654]

Note that if the result of List Comprehensions is more than one member at a time, the result values ​​must be enclosed in parentheses (as a tuple object).

Consider the example [x, y) for x in a for y in b if x> 0)] and its output. Given this, the following statement is incorrect in the opinion of the Python interpreter:

>>> [x,y for x in a for y in b]
  File "<stdin>", line 1
    [x,y for x in a for y in b]
           ^
SyntaxError: invalid syntax
>>>

Another important point remains. Note the example of the code below in the two versions of Python 3x and 2x:

# Python 3.x

>>> x = 'before'

>>> a = [x for x in (1, 2, 3)]
>>>

>>> x
'before'

# Python 2.x

>>> x = 'before'

>>> a = [x for x in (1, 2, 3)]
>>>

>>> x
3

Both codes are the same, but in version 2x, because the iteration variables defined – here x – are not considered in a separate scope, by changing their value inside the expression, the value of the same name in the outer field of the expression also changes. It will be given. According to Mr. Rossom, “dirty little secret” has been fixed in version 3x. [more details]

Generator Expressions

The function of Generator Expressions is similar to List Comprehensions, but with the property of a Generator object, and to create it, it is enough to use parentheses () instead of brackets [] in List Comprehensions. [PEP 289]:

>>> a = [1, 2, 3, 4]

>>> b = (10*i for i in a)
>>>
>>>

>>> b
<generator object <genexpr> at 0x7f488703aca8>
>>>

>>> b.__next__()  # python 3.x - In Python 2.x use 
.next()
10

>>> b.__next__()  # python 3.x - In Python 2.x use 
.next()
20
>>>

Understanding the difference between Generator Expressions and List Comprehensions is very important. The output of a List Comprehensions is exactly the result of performing operations in the form of a list object, while the output of a Generation Expressions is an object that knows how to produce results step by step. Understanding such issues will play an important role in increasing program performance and memory consumption.

By running the sample code below; Out of all the rows in the The_Zen_of_Python.txt file, the rhyming comments are printed in Python:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
------------------------------------------------------------------
# File Name: The_Zen_of_Python.txt
# The Zen of Python
# PEP 20: https://www.python.org/dev/peps/pep-0020

>>> file = open("/home/saeid/Documents/The_Zen_of_
Python.txt")

>>> lines = (t.strip() for t in file)

>>> comments = (t for t in lines if t[0] == '#')

>>> for c in comments:
...     print(c)
...
# File Name: The_Zen_of_Python.txt
# The Zen of Python
# PEP 20: https://www.python.org/dev/peps/pep-0020
>>>

In the first row, the The_Zen_of_Python.txt file is opened, and in the second row, a Generator object is obtained to access and strip them (remove possible space characters at the beginning and end of the line text) using Generator Expressions. Note that the lines of the file have not been read yet, and only the ability to request and navigate line by line is created. In the third line, by creating another Generator object (again in the Generator Expressions method), we have the ability to filter comment-like lines inside the file with the help of the lines object of the previous step. But the rows of the file have not been read yet because a production request has not yet been sent to either of the generator objects (lines and comments). Finally, in the fourth line, the for loop command executes the comments object, and this object also executes the lines object based on the operations defined for it.

The_Zen_of_Python.txt file used in this example is very small, but you can see the effect of using Generator Expressions in this example by extracting comments from a multi-gigabyte file!

tip

The Generator object created in the Generator Expressions method can also be converted to a list object using the list () function:

>>> comment_list = list(comments)

>>> comment_list
['# File Name: The_Zen_of_Python.txt',
'# The Zen of Python',
'# PEP 20: https://www.python.org/dev/peps/pep-0020']

lambda and anonymous functions

In the Python programming language, Anonymous functions or Lambda functions are functions that can have any number of arguments, but their body must contain only one expression. The lambda keyword is used to construct these functions. The structural pattern of this type of function is as follows:

lambda args : expression

In this template, args represents any number of arguments separated by commas (,), and the expression represents only one Python expression that does not contain statements such as for or while.

Consider the following function as an example:

>>> def a_function(x, y):
...     return x + y
...
>>>

>>> a_function(2, 3)
5

This function will be in anonymous form as follows:

>>> a_function = lambda x,y : x+y

>>> a_function(2, 3)
5

Or:

>>> (lambda x,y: x+y)(2, 3)
5

Where is the main application of Lambda functions?

These functions are mostly used when we want to pass a short function as an argument to another function.

For example, in Lesson 8, we remember that the sort () method was used to sort the members of a list object, and it was stated that the sort () method has an optional argument called key, which can be passed by passing a single-argument function to it. Each member of the list did this before comparing and sorting (for example: capitalizing):

>>> L = ['a', 'D', 'c', 'B', 'e', 'f', 'G', 'h']

>>> L.sort()

>>> L
['B', 'D', 'G', 'a', 'c', 'e', 'f', 'h']

>>> L.sort(key=lambda n: n.lower())

>>> L
['a', 'B', 'c', 'D', 'e', 'f', 'G', 'h']
>>>

😊 I hope it was useful

This lesson introduces the concept of function in the Python programming language.

✔ Level: Medium

Headlines

Lesson 12: Function in Python: Defining, Submitting, and Matching

Arguments
Introduction
Syntax
Name and domain spaces
Submit an argument
Matching arguments

Introduction

A function is a block of commands that need to be called to execute, and this call can be performed more than once. The function can receive values ​​when called and return a value as a result if necessary.

It is an alternative function for duplicate parts of the program, which can be avoided by changing the program once and writing it several times and calling and executing it. Using functions increases the reusability of code and reduces redundancy. Functions are tools to break program logic into smaller executives to facilitate the construction of large programs.

The concept of function is introduced into mathematics from programming and means a tool that can calculate the result (or output) based on input data. If all the basic parts of the program are developed with such an approach, this method of development is called “Functional Programming”, the way you see it in programming languages ​​like Scala, Scheme, Lisp and Haskell. Functions are also available in other languages ​​such as C, Pascal, and C ++, although these languages ​​fall into the “imperative programming” category, and their approach to the function is quite different from what exists in functional programming languages. has it. Explaining the difference between the two is a bit difficult, but you will surely find out by programming with the languages ​​in these two categories. 🙂

The approach of this tutorial to the function is based on declarative programming. The Python language also offers functional programming capabilities, along with other approaches such as declaration or object-orientation. For functional programming using Python, see Functional Programming in Python by O’Reilly (download pdf – free).

In addition to the term function in programming, there is another similar term as “procedure”. Procedures and functions are similar in structure, except that procedures do not return values. In Python, there is no syntax for defining routines, but functions that do not return a value are also called routines because the functions in Python return a value under any circumstances, even if this value is None.

Syntax

Functional Syntax in Python Programming Language Like any other compound command, it contains a header and a body – Lesson 6. The header section contains the def keyword, a user-preferred name, and a parenthesis, which indicates where the function parameters are located. Each function can accept any, one or more parameters:

def function_name(param1, param2,... paramN):
    statements

As has been said many times, everything is an object in Python, when the program execution reaches the def keyword, first a function object is created and then it is referenced by the function name (here: function_name):

>>> def func_name():
...     pass
...
>>>

>>> type(func_name)
<class 'function'>

tip

PEP 8 Recommendation: The function name should consist of lowercase letters whose words are separated by underscores. Such as: my_function. MixCase mode such as: myFunction is also valid as long as the function names are written with the same pattern throughout the code.

The body will not run until the function is called. The function name + parentheses is used to call the function, and if parameters are included in the function definition, arguments corresponding to these parameters must be sent when calling:

function_name(arg1, arg2,... argN)

consideration

In the discussion of functions, the variables that are defined in the function header are called parameters (Parameter) and the data that is sent when the function is called is called Argument. For each parameter considered in the function definition, a similar argument must be passed to it. In fact, arguments are values ​​that we want to assign to the parameters of a function. There is no need for similar arguments and parameters to be synonymous, but the existence of a synonym makes the code more readable.

The body of the function can also contain the return keyword. In fact, return is a command that is given anywhere in the body, the execution of the function stops at that point and returns a value (in Python, of course, it is correct to say: object) as a result to the function call:

def function_name(param1, param2,... paramN):
    ...
    return value

In the example code above, value is the value returned by the return to the calling location. value can not be explicitly a value but an expression such as: param1 ** 2 or param1> 3, etc., in which case the result of the expression is evaluated first and then returned. If no value is specified, None will be returned:

>>> def my_pow(x, y):
...     return x**y
...
>>>

>>> a = 2

>>> b = 3
>>>

>>> my_pow(a, b)
8
>>>

tip

If the return command is not written at the end of the function, the Python interpreter implicitly considers the return None command. Therefore, by calling such functions in the Python language, the value None will be returned (albeit a rather boring one) after the instructions inside the body are fully executed.

tip

Recommendation PEP 8: Operate uniformly to return the value. If you use compound statements such as if / else, either none of the sections should explicitly return, or if at least one section needs to return a value, the rest of the sections should return a value, even if this value is None. To be taken:

YES:

def foo(x):
    if x >= 0:
        return math.sqrt(x)
    else:
        return None


NO:

def foo(x):
    if x >= 0:
        return math.sqrt(x)

 In the Python programming language, a function is a “first-class” entity, which means that a function can be dynamically created or destroyed like other objects, passed as an argument to other functions, returned as a result, and so on. The result can be defined as a function inside the body of control statements (while, if, etc.) or inside the body of another function:

>>> def outer(num1):
...     def inner_increment(num1):  
# hidden from outer code
...         return num1 + 1
...     num2 = inner_increment(num1)
...     print(num1, num2)
...
>>>

>>> outer(1)
1 2

It’s a good idea to use “Docstring” in functions to help better document and read the program – Lesson 6:

def function_with_docstring(param1, param2):
    """Example function with types documented in the 
docstring.

    Args:
        param1 (int): The first parameter.
        param2 (str): The second parameter.

    Returns:
        bool: The return value. True for success, 
False otherwise.
    """

Name and domain spaces

There are many names in each Python program, for example: variables, function names, class names, and so on. Obviously, letters need to be unique to identify objects, which is difficult to do in an even small program. In Python, there is a structure called “Namespaces” to categorize and prevent letters from interfering. Each space contains the name of a part of the letters in the program. In general, Python namespaces are depicted on three levels: “Local”, “Global” and Built-in:

Each Python module forms a global space name that is isolated from other module names. Spatial Name All modules are created within a larger spatial space known as the Built-in space, and the names of all the ready-made functions such as open () that we used to use are located in this space. Your structure within the namespace enables us to access built-in functions anywhere in the program without having to import a specific module.

Each module can contain a number of functions and classes. By calling each function, a local space name for that function is created within the space module of the corresponding module, and when the function is executed, it disappears. The same thing happens with classes. Accordingly, we can create different variables inside the function, but with the same name as the variables outside the function in the module, because they are located in two different space names. The function also had access.

We said that space is the name of the modules isolated from each other. Therefore, to access the names in other modules, we must first import those modules, in which case they can be accessed using the module name – as a prefix. For example, obtaining the name getcwd, which refers to a function from the OS module space, is shown in the example code below:

>>> import os

>>> os.getcwd()
'/home/saeid'

But what about using the letters of a module within itself? Where there are other namespaces such as functions, but there is no prefix such as module name that can distinguish letters within these different spaces. To understand how each module name is accessed anywhere in the same module, we are introduced to another concept called “Scope“. Generally, domains are areas of the program where a name can be used without the use of any prefix and without interfering with other names. The domain discussion is only within each module.

Field rules:

• The body of a module – meaning the areas outside the body of functions and classes – is the Global Scope. Note that the word “global” in the domain (or space name) discussion only refers to all the code inside each module, not the entire program. In general, remember the module wherever you hear (or read) the word Global in the Python language:

# This is a global variable
a = 0

if a == 0:
    # This is still a global variable
    b = 1

In the code example above, the domain definition for both variables a and b is global. The body of control commands does not have a separate space name, and the definition of the variable in these areas of the program falls within the global domain.

• The body of each function is a Local Scope, and by default all variables generated within the functions are located within the local domain unless specified using global or nonlocal keywords. If we want to do within the attribution function to one of the names in the global domain, we must use the global command. Note the sample codes below:

def my_function(c):
    # this is a local variable
    d = 3

>>> a = 0
>>>

>>> def my_function():
...    a = 3
...    print(a)
...
>>>

>>> a
0

>>> my_function()
3

>>> a
0
>>>

>>> a = 0
>>>

>>> def my_function():
...     global a
...     a = 3
...     print(a)
...
>>>

>>> a
0

>>> my_function()
3

>>> a
3
>>>

It does not make a difference in nested functions either, each function that is called creates a separate space for it and will have its own local domain. The nonlocal command is provided in Python 3 and is used in nested functions. When we want to assign an internal function inside the body to a name defined in one of its external functions, we must use this command to specify the desired name:

>>> def outer():
...     x = 1
...     def inner():
...         x = 2
...         print("inner:", x)
...     inner()
...     print("outer:", x)
...
>>>

>>> outer()
inner: 2
outer: 1
>>>

>>> def outer():
...     x = 1
...     def inner():
...         nonlocal x
...         x = 2
...         print("inner:", x)
...     inner()
...     print("outer:", x)
...
>>>

>>> outer()
inner: 2
outer: 2
>>>

• When using a variable, the Python interpreter must first identify its domain and space name in order to find the object to which the variable refers. Suppose a variable inside a phrase is used in the body of a function. It goes on to finally examine the global scope of the module and then the Built-in; If no result, a NameError exception occurs:

>>> x = 0
>>>

>>> def outer():
...     x = 1
...     def inner():
...         print(x)
...     inner()
...
>>> outer()
1

>>> x = 0
>>>

>>> def outer():
...     def inner():
...         print(x)
...     inner()
...
>>> outer()
0

>>> x = 0
>>>

>>> def outer():
...     def inner():
...         print(z)
...     inner()
...
>>> outer()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in outer
  File "<stdin>", line 3, in inner
NameError: name 'z' is not defined
>>>

Submit an argument

By automatically submitting an argument to the function, local variables are created by assigning argument objects to the names of the parameters in the function header:

>>> def f(a):
...     print(a*a)
...
>>>

>>> b = 3

>>> f(b)
9

By calling the function f in the above code instance, the local variable a is created, which refers to the correct object 3.

Note that by assigning a new object to the function parameters, sending an argument is practically ineffective:

>>> def f(a):
...     a = 2
...     print(a*a)
...
>>> b = 3

>>> f(b)
4

The important thing in submitting an argument is to pay attention to how it is!

Among programming languages, there are two common ways to send an argument: “by value” and “by reference“. In the by value method, a copy of the value of the argument is sent to the function, and as a result, by changing the value of the corresponding parameter in the function, the value of the argument sent outside the function remains unchanged. Consider the Python example below:

>>> def f(a):
...     a = 2
...     print(a*a)
...
>>> b = 3

>>> f(b)
4

>>> b
3

As can be seen in the code example above, the value of variable b remains unchanged.

But in the by reference method, instead of sending a copy of the value of the argument, a reference is sent from the argument to the function. It can be assumed that the corresponding parameter in the function is the same argument outside the function. As a result, by changing the value of the corresponding parameter in the function, the value of the argument outside the function also changes. Consider the Python example below:

>>> def f(a):
...    a[0] = 3
...    print(a)
...
>>> b = [1, 2]

>>> f(b)
[3, 2]

>>> b
[3, 2]

These are two common methods in programming languages, but what is it like to send a parameter specifically in the Python programming language? In Python, everything is an object, and as a result, arguments are passed “by reference” in any situation.

And if the question is what is the reason for the difference in behavior in the previous two examples? We need to know that the reason is related to the nature of the submitted argument objects. Sending immutable objects such as boolean, numbers, strings, and tuple types to the function will cause behavior similar to the by value method, but not sending mutable objects such as list, dictionary, and collection types. Note the pictures below:

Variable objects in Python are objects whose value can be changed without changing their id (). The output of the id‍‍ () function for each object represents the unique identifier of that object, which is actually its address in memory [Python Documents] – Lesson Five.

To prevent changeable objects within the function, you can create a copy of this type of object as described in Lesson 8 and then send it to the function as an argument:

>>> def f(a):
...     a[0] = 3
...     print(a)
...

>>> b = [1, 2]

>>> f(b[:])      # Pass a copy
[3, 2]

>>> b
[1, 2]

In the example code above, since all members of the variable b list object are all immutable types, a Shallow Copy of the object is sufficient, but otherwise a Deep Copy of the object must be sent – Lesson Eight.

Of course, sometimes we really need to have the modified value of the variable sent to the function outside the function as well. For this purpose, in some programming languages, it is possible to send by reference according to the request of the programmer. For example in php this is done by putting an & after the desired parameter:

<?php
function foo(&$var)
{
    $var++;
}

$a=5;
foo($a);
// $a is 6 here
?>

There is no such feature in Python, at least for immutable objects! But it can be covered using the ability to return multiple objects with the return command. Using this method, any number of required parameters can be transferred out of the function:

>>> def multiple(x, y):
...     x = 2
...     y = [3, 4]
...     return x, y
...

>>> X = 1

>>> Y = [1, 2]
>>>

>>> X, Y = multiple(X, Y)
>>>

>>> X
2

>>> Y
[3, 4]

Note that in this case, the return statement returns all of these objects in the form of a tuple object:

>>> multiple(X, Y)
(2, [3, 4])

Matching arguments

Earlier it was mentioned that the number of arguments submitted must match the parameters in the function header:

>>> def f(a, b, c):
...     pass
...
>>>

>>> f(1, 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required positional argument: 
'c'
>>>

>>> f(1, 2, 3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() takes 3 positional arguments but 
4 were given
>>>

In the following, we present the types of acceptable Python syntaxes in matching arguments (Argument Matching) with function parameters:

• The usual syntax we have used so far is to explicitly send a unique argument for each parameter. The matching operation in this syntax is based on the position of the arguments, so the order of the arguments must correspond to the order of the parameters in the function header:

>>> def f(a, b, c):
...     print("a= ", a)
...     print("b= ", b)
...     print("c= ", c)
...

>>> f(1, 2, 3)
a=  1
b=  2
c=  3

>>> f("one", 2, [3,33,333])
a=  one
b=  2
c=  [3, 33, 333]
>>>

• Name Syntax = Value In this syntax, arguments are assigned to the name of the parameters, and since the matching operation is based on the name of the parameters, the position or order of the arguments no longer matters:

>>> def f(a, b, c):
...     print(a, b, c)
...

>>> f(a=1, c=3, b=2)
1 2 3

These two syntaxes can also be used in combination. It is only necessary to note that the arguments whose matching action depends on the situation – in the order of the previous cases – first. Consider the example below:

>>> def f(a, b, c):
...     print("a= ", a)
...     print("b= ", b)
...     print("c= ", c)
...

>>> f(1, c=3, b=2)
a=  1
b=  2
c=  3

>>> f(1, 2, c=3)
a=  1
b=  2
c=  3
>>>

For the example function above, the following call modes are incorrect:

>>> f(c=3, b=2, 1)
  File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument

>>> f(a=1, 2, c=3)
  File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument

>>> f(a=1, 2, 3)
  File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument

>>> f(2, a=1, c=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for argument 'a'

• Iterable * syntax In this syntax, an object of iterable type (iterable – lesson 9) such as strings, tuples, lists, etc., which is marked by an asterisk * * character, is sent to the function. In this case, based on the position order, the members inside the repeatable object are assigned to the function parameters:

>>> def f(a, b, c):
...     print("a= ", a)
...     print("b= ", b)
...     print("c= ", c)
...
>>>

>>>my_list = [1, 2, 3]
>>>

>>> f(*my_list)
a=  1
b=  2
c=  3
>>>

>>> my_list_2 = [1, 2, 3, 4]
>>>

>>> f(*my_list_2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() takes 3 positional arguments but 
4 were given
>>>

• Syntax dict ** In this syntax, a dictionary object marked with two asterisk ** characters is sent to the function. The keys of this dictionary object must have the same name as the parameters defined in the function header. After calling the function, this dictionary object opens and based on the name of the key in the key pairs: the value inside it, the parameters of the function are set:

>>> def f(a, b, c):
...     print(a, b, c)
...

>>> b = {'a':1, 'c':3, 'b':2}

>>> f(**b)
1 2 3

These four syntaxes discussed the determination of arguments when calling a function, and in all of them the number of arguments submitted must be equal to the number of parameters defined in the function header. The value of the dictionary object is equal to the number of function parameters.

In the following, we will present the syntaxes in this field that are involved in determining the function parameters.

• The usual syntax we have used so far is to explicitly define each parameter:

>>> def f(a, b, c):
...     print(a, b, c)
...

>>> f(1, 2, 3)
1 2 3

• Syntax Set the default value for the parameters. When defining any parameter in the function definition, a value can also be assigned to it; In this case, if a similar argument is not sent with that parameter, the default value of that parameter will be considered. Such parameters are also called optional:

>>> def chaap(text=None):
...     if text:
...         print(text)
...     else:
...         print("Nothing!")
...
>>>

>>> chaap("Python :)")
Python :)
>>>

>>> chaap()
Nothing!
>>>

The parameter with the default value can be defined next to the mandatory parameters (without the default value), in which case the parameters with the default value must be placed at the end:

>>> def f(a, b=2, c=3):     
# a required, b and c optional
        print(a, b, c)

>>> f(1)          # Use defaults
1 2 3

>>> f(a=1)
1 2 3

>>> f(1, 4)       # Override defaults
1 4 3

>>> f(1, 4, 5)
1 4 5

>>> f(1, c=6)     # Choose defaults
1 2 6

• The syntax name * receives all submitted arguments in the form of a topple object – this feature is very helpful when the number of submitted arguments varies:

>>> def f(*name):
...     print(type(name))
...     print(name)
...
>>>

>>> f(1)
<class 'tuple'>
(1,)

>>> f(1, 2, 3)
<class 'tuple'>
(1, 2, 3)

>>> def f(a, b=2, *args):
...     print("a= ", a)
...     print("b= ", b)
...     print("args= ", args)
...
>>>

>>> f(1)
a= 1
b= 2
args= ()


>>> f(1, 5)
a= 1
b= 5
args= ()


>>> f(1, "FF", 3)
a= 1
b= FF
args= (3,)


>>> f(1, "FF", 3, 4, 5)
a= 1
b= FF
args= (3, 4, 5)


>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required positional argument: 
'a'

>>> a_list = [3, (4, 5)]
>>>

>>> f(a_list)
a= [3, (4, 5)]
b= 2
args= ()

>>> f(1, 4, [8, 12, 16])
a= 1
b= 4
args= ([8, 12, 16],)

>>> a_list = [3, 6, 9, (10, 11)]
>>>

>>> f(*a_list)
a= 3
b= 6
args= (9, (10, 11))

>>> def f(a, *b, c):
...     print(a, b, c)
...

>>> f(a=1, b=2, c=3)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() got an unexpected keyword argument 'b'
>>>

Note that the argument cannot be passed to the starred parameter using the name = value method.

Keyword-Only Arguments PEP 3102

In Python version 2x, after the starred parameter, no other parameter can be set, and this is only possible in Python version 3x. It should be noted that in this version (3x) the arguments, like all parameters that are placed after the starred parameter, must be sent as name = value.

Consider a function with a header def f (a, * b, c.) In such a case where parameter a is set using its position and parameter b receives any number of other arguments, it is no longer necessary to pass the argument to parameter c. There is no choice but to mention its name!

>>> def f(a, *b, c):
...     print(a, b, c)
...
>>>

>>> f(1, 2, c=3)
1 (2,) 3

>>> f(1, c=3)
1 () 3

>>> f(1, 2 ,3 ,4 , 5, c=30)
1 (2, 3, 4, 5) 30


>>> f(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required keyword-only 
argument: 'c'


>>> f(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required keyword-only 
argument: 'c'


>>> f(1, 2 ,3 ,4 , 5, c=30, 40)
  File "<stdin>", line 1
SyntaxError: positional argument follows 
keyword argument

>>> def f(a, *b, c, d=5):
...     print(a, b, c, d)
...
>>>

>>> f(1, 2, 3, c=4)
1 (2, 3) 4 5

>>> f(1, 2, 3, 4, c=7, d=9)
1 (2, 3, 4) 7 9

>>> # Python 2.x

>>> def f(a, *b, c):
  File "<stdin>", line 1
    def f(a, *b, c):
                 ^
SyntaxError: invalid syntax

• The syntax name ** receives all the key arguments: the value sent in the form of a dictionary object:

>>> def f(**name):
...     print(type(name))
...     print(name)
...
>>>

>>> f()
<class 'dict'>
{}

>>> f(a=1)
<class 'dict'>
{'a': 1}

>>> def f(a, b=2, **kwargs):
...     print(a, b, kwargs)
...
>>>

>>> f(1, c=3)
1 2 {'c': 3}

>>> f(b=10, a=5, c=15)
5 10 {'c': 15}

>>> def f(a, b=2, *args, **kwargs):
...     print(a, b , args, kwargs)
...
>>>

>>> f(*[1, 2, 3, 4 ,5], **{"c":7, "d":9})
1 2 (3, 4, 5) {'d': 9, 'c': 7}

>>> f(11, 12, 13, 14, 15)
11 12 (13, 14, 15) {}

>>> f(b=14, a=7, c=21, d=28)
7 14 () {'d': 28, 'c': 21}

😊 I hope it was useful

The Python Standard Library has a wide range of ready-made features provided by installing Python. You can see the full list of these features at (Python 2x) and (Python 3x). It should be noted that a large part of Python’s strength is due to its many powerful libraries, many of which are being developed outside the standard Python library and within the user community, and an almost complete list of which can also be searched and downloaded by PyPI.

This lesson, as the last lesson from the introductory level of the book, is dedicated to examining some of the practical possibilities of this library, which, of course, we may have used during the previous lessons!

✔ Level: Introductory

Headlines

Lesson 11: Python Standard Library: math and os

math os os.path

The subject of this lesson is how to receive or read data from an interactive user or files, as well as display or write to them; Accordingly, the text of the lesson will examine the file object and the print () and input () functions in Python. Some differences in the implementation of the new version of Python (3x branch) that are related to the subject of this lesson are also mentioned.

✔ Level: Introductory

Headlines

Lesson 10: File, input and output in Python

Files Standard file object Input () function Print () function

Normally, the execution flow of a program has a fixed process in which the code is read and executed line by line, from top to bottom; Control commands are the ability to control or change this constant current. Using these commands, you can set a condition for the execution of a block that if this condition is not met at the time of execution, the execution of the block will be canceled or conditions can be created for the execution of a block to be selected from several specific blocks. The execution of a block can be repeated several times. This lesson is devoted to examining Python control commands in two sections, “Selection” and “Repetition”.

✔ Level: Introductory

Headlines

Lesson 09: Control commands in Python

Selection Repetition

Selection

Using the selection command, you can determine whether a command block will be executed or not, and which of the two or more command blocks will be selected and executed according to the program conditions at runtime. Python offers only one selection structure that can be implemented in three forms: “single-choice”, “two-choice” and “multi-choice”; This structure is called the if statement and will be discussed below.

Python represents each “Data Type” by a class; So each data is a sample or object of a specific class. Although the programmer can also define the class of his choice, in this lesson we want to talk about that part of the data types or types of objects that are provided to the Python interpreter in a ready-made (Built-in) way.

This lesson will only look at “numeric object types” and “string types” in Python, and the remaining types will be explored in the next lesson. Although efforts have been made to provide complete details, in some areas, reading Python official documents can provide you with more complete information. Many Python-ready functions will be used, which may be more detailed than what is described in this lesson; For this reason, a link to their definition is also provided in the Python documentation. The important point in studying this lesson is to examine the sample code, which sometimes it will be impossible to understand the explanation given without accuracy in this sample code.

✔ Level: Introductory

Headlines

Lesson 07: Data Types or Objects in Python: Part 1

• True
• Floating point
• mixed
• December
• Kasr
• boolean
• field

This lesson introduces the basic components of Python programming and examines what they are, what their grammar is, what they do, and so on. There are also many instances of the standard Python scripting method recommended to Python programmers by the PEP 8 document ; Adherence to these principles contributes to the uniformity of Python community codes.

Syntax ( Syntax or syntax) is a set of rules that defines how to program in a language specified; For example, how a text is written to be interpreted by a Python interpreter or a string object depends on the syntax defined in Python, and an exception will be reported if the interpreter fails to conform to any of the defined rules. Python syntax is not limited to this lesson, and you will see many other things, such as how to define different objects, in future lessons.

✔ Level: Introductory