translation: Update the complexity analysis chapter to the main branch (#1001)
* Update buttons. * Update button svg * Update button * Update README.md * Update index.md * Update translation of about_the _book * Update English headings. * Update the translation of chapter comlexity analysis to main branch.pull/1003/head
Yudong Jin
11 months ago
committed by
GitHub
parent
6d961d38a0
commit
9d9d9e74b9
@ -1,194 +1,194 @@
|
||||
# Iteration vs. Recursion
|
||||
# Iteration and Recursion
|
||||
|
||||
In data structures and algorithms, it is common to repeat a task, which is closely related to the complexity of the algorithm. There are two basic program structures that we usually use to repeat a task: iteration and recursion.
|
||||
In algorithms, repeatedly performing a task is common and closely related to complexity analysis. Therefore, before introducing time complexity and space complexity, let's first understand how to implement task repetition in programs, focusing on two basic programming control structures: iteration and recursion.
|
||||
|
||||
## Iteration
|
||||
|
||||
An "iteration iteration" is a control structure that repeats a task. In iteration, a program repeats the execution of a piece of code until the condition is no longer satisfied.
|
||||
"Iteration" is a control structure for repeatedly performing a task. In iteration, a program repeats a block of code as long as a certain condition is met, until this condition is no longer satisfied.
|
||||
|
||||
### For Loops
|
||||
### for Loop
|
||||
|
||||
`for` loops are one of the most common forms of iteration, **suitable when the number of iterations is known in advance**.
|
||||
The `for` loop is one of the most common forms of iteration, **suitable for use when the number of iterations is known in advance**.
|
||||
|
||||
The following function implements the summation $1 + 2 + \dots + n$ based on a `for` loop, and the result is recorded using the variable `res`. Note that `range(a, b)` in Python corresponds to a "left-closed-right-open" interval, which is traversed in the range $a, a + 1, \dots, b-1$.
|
||||
The following function implements the sum $1 + 2 + \dots + n$ using a `for` loop, with the sum result recorded in the variable `res`. Note that in Python, `range(a, b)` corresponds to a "left-closed, right-open" interval, covering $a, a + 1, \dots, b-1$:
|
||||
|
||||
```src
|
||||
[file]{iteration}-[class]{}-[func]{for_loop}
|
||||
```
|
||||
|
||||
The figure below shows the flow block diagram of this summation function.
|
||||
The flowchart below represents this sum function.
|
||||
|
||||
|
||||
|
||||
|
||||
The number of operations in this summation function is proportional to the size of the input data $n$, or a "linear relationship". In fact, **time complexity describes this "linear relationship"**. This is described in more detail in the next section.
|
||||
The number of operations in this sum function is proportional to the input data size $n$, or in other words, it has a "linear relationship". This is actually what **time complexity describes**. This topic will be detailed in the next section.
|
||||
|
||||
### While Loop
|
||||
### while Loop
|
||||
|
||||
Similar to a `for` loop, a `while` loop is a way to implement iteration. In a `while` loop, the program first checks the condition at each turn, and if the condition is true, it continues, otherwise it ends the loop.
|
||||
Similar to the `for` loop, the `while` loop is another method to implement iteration. In a `while` loop, the program checks the condition in each round; if the condition is true, it continues, otherwise, the loop ends.
|
||||
|
||||
Below, we use a `while` loop to realize the summation $1 + 2 + \dots + n$ .
|
||||
Below we use a `while` loop to implement the sum $1 + 2 + \dots + n$:
|
||||
|
||||
```src
|
||||
[file]{iteration}-[class]{}-[func]{while_loop}
|
||||
```
|
||||
|
||||
In `while` loops, since the steps of initializing and updating condition variables are independent of the loop structure, **it has more degrees of freedom than `for` loops**.
|
||||
**The `while` loop is more flexible than the `for` loop**. In a `while` loop, we can freely design the initialization and update steps of the condition variable.
|
||||
|
||||
For example, in the following code, the condition variable $i$ is updated twice per round, which is not convenient to implement with a `for` loop.
|
||||
For example, in the following code, the condition variable $i$ is updated twice in each round, which would be inconvenient to implement with a `for` loop:
|
||||
|
||||
```src
|
||||
[file]{iteration}-[class]{}-[func]{while_loop_ii}
|
||||
```
|
||||
|
||||
Overall, **`for` loops have more compact code and `while` loops are more flexible**, and both can implement iteration structures. The choice of which one to use should be based on the needs of the particular problem.
|
||||
Overall, **`for` loops are more concise, while `while` loops are more flexible**. Both can implement iterative structures. Which one to use should be determined based on the specific requirements of the problem.
|
||||
|
||||
### Nested Loops
|
||||
|
||||
We can nest one loop structure inside another, using the `for` loop as an example:
|
||||
We can nest one loop structure within another. Below is an example using `for` loops:
|
||||
|
||||
```src
|
||||
[file]{iteration}-[class]{}-[func]{nested_for_loop}
|
||||
```
|
||||
|
||||
The figure below gives the block diagram of the flow of this nested loop.
|
||||
The flowchart below represents this nested loop.
|
||||
|
||||
|
||||
|
||||
|
||||
In this case, the number of operations of the function is proportional to $n^2$, or the algorithm's running time is "squared" to the size of the input data $n$.
|
||||
In this case, the number of operations in the function is proportional to $n^2$, or the algorithm's running time and the input data size $n$ have a "quadratic relationship".
|
||||
|
||||
We can continue to add nested loops, and each nest is a "dimension up", which will increase the time complexity to "cubic relations", "quadratic relations", and so on.
|
||||
We can continue adding nested loops, each nesting is a "dimensional escalation," which will increase the time complexity to "cubic," "quartic," and so on.
|
||||
|
||||
## Recursion
|
||||
|
||||
"Recursion recursion is an algorithmic strategy to solve a problem by calling the function itself. It consists of two main phases.
|
||||
"Recursion" is an algorithmic strategy that solves problems by having a function call itself. It mainly consists of two phases.
|
||||
|
||||
1. **recursive**: the program calls itself deeper and deeper, usually passing smaller or simpler arguments, until a "termination condition" is reached.
|
||||
2. **Recursion**: After the "termination condition" is triggered, the program returns from the deepest level of the recursion function, level by level, aggregating the results of each level.
|
||||
1. **Recursion**: The program continuously calls itself, usually with smaller or more simplified parameters, until reaching a "termination condition."
|
||||
2. **Return**: Upon triggering the "termination condition," the program begins to return from the deepest recursive function, aggregating the results of each layer.
|
||||
|
||||
And from an implementation point of view, recursion code contains three main elements.
|
||||
From an implementation perspective, recursive code mainly includes three elements.
|
||||
|
||||
1. **Termination condition**: used to decide when to switch from "recursive" to "inductive".
|
||||
2. **Recursion call**: corresponds to "recursion", where the function calls itself, usually with smaller or more simplified input parameters.
|
||||
3. **return result**: corresponds to "return", returning the result of the current recursion level to the previous one.
|
||||
1. **Termination Condition**: Determines when to switch from "recursion" to "return."
|
||||
2. **Recursive Call**: Corresponds to "recursion," where the function calls itself, usually with smaller or more simplified parameters.
|
||||
3. **Return Result**: Corresponds to "return," where the result of the current recursion level is returned to the previous layer.
|
||||
|
||||
Observe the following code, we only need to call the function `recur(n)` , and the calculation of $1 + 2 + \dots + n$ is done:
|
||||
Observe the following code, where calling the function `recur(n)` completes the computation of $1 + 2 + \dots + n$:
|
||||
|
||||
```src
|
||||
[file]{recursion}-[class]{}-[func]{recur}
|
||||
```
|
||||
|
||||
The figure below shows the recursion of the function.
|
||||
The figure below shows the recursive process of this function.
|
||||
|
||||
|
||||
|
||||
|
||||
Although iteration and recursion can yield the same results from a computational point of view, **they represent two completely different paradigms for thinking about and solving problems**.
|
||||
Although iteration and recursion can achieve the same results from a computational standpoint, **they represent two entirely different paradigms of thinking and solving problems**.
|
||||
|
||||
- **Iteration**: solving problems "from the bottom up". Start with the most basic steps and repeat or add to them until the task is completed.
|
||||
- **Recursion**: solving problems "from the top down". The original problem is broken down into smaller subproblems that have the same form as the original problem. Next, the subproblem continues to be broken down into smaller subproblems until it stops at the base case (the solution to the base case is known).
|
||||
- **Iteration**: Solves problems "from the bottom up." It starts with the most basic steps, then repeatedly adds or accumulates these steps until the task is complete.
|
||||
- **Recursion**: Solves problems "from the top down." It breaks down the original problem into smaller sub-problems, each of which has the same form as the original problem. These sub-problems are then further decomposed into even smaller sub-problems, stopping at the base case (whose solution is known).
|
||||
|
||||
As an example of the above summation function, set the problem $f(n) = 1 + 2 + \dots + n$ .
|
||||
Taking the sum function as an example, let's define the problem as $f(n) = 1 + 2 + \dots + n$.
|
||||
|
||||
- **Iteration**: the summation process is simulated in a loop, iterating from $1$ to $n$ and executing the summation operation in each round to find $f(n)$.
|
||||
- **Recursion**: decompose the problem into subproblems $f(n) = n + f(n-1)$ and keep (recursively) decomposing until the base case $f(1) = 1$ terminates.
|
||||
- **Iteration**: In a loop, simulate the summing process, iterating from $1$ to $n$, performing the sum operation in each round, to obtain $f(n)$.
|
||||
- **Recursion**: Break down the problem into sub-problems $f(n) = n + f(n-1)$, continuously (recursively) decomposing until reaching the base case $f(1) = 1$ and then stopping.
|
||||
|
||||
### Call The Stack
|
||||
### Call Stack
|
||||
|
||||
Each time a recursion function calls itself, the system allocates memory for the newly opened function to store local variables, call addresses, other information, and so on. This results in two things.
|
||||
Each time a recursive function calls itself, the system allocates memory for the newly initiated function to store local variables, call addresses, and other information. This leads to two main consequences.
|
||||
|
||||
- The context data for a function is stored in an area of memory called "stack frame space" and is not freed until the function returns. As a result, **recursion is usually more memory-intensive than iteration**.
|
||||
- Recursion calls to functions incur additional overhead. **Therefore recursion is usually less time efficient than loops**.
|
||||
- The function's context data is stored in a memory area called "stack frame space" and is only released after the function returns. Therefore, **recursion generally consumes more memory space than iteration**.
|
||||
- Recursive calls introduce additional overhead. **Hence, recursion is usually less time-efficient than loops**.
|
||||
|
||||
As shown in the figure below, before the termination condition is triggered, there are $n$ unreturned recursion functions at the same time, **with a recursion depth of $n$** .
|
||||
As shown in the figure below, there are $n$ unreturned recursive functions before triggering the termination condition, indicating a **recursion depth of $n$**.
|
||||
|
||||
|
||||
|
||||
|
||||
In practice, the depth of recursion allowed by a programming language is usually limited, and too deep a recursion may result in a stack overflow error.
|
||||
In practice, the depth of recursion allowed by programming languages is usually limited, and excessively deep recursion can lead to stack overflow errors.
|
||||
|
||||
### Tail Recursion
|
||||
|
||||
Interestingly, **if a function makes a recursion call only at the last step before returning**, the function can be optimized by the compiler or interpreter to be comparable to iteration in terms of space efficiency. This situation is called "tail recursion tail recursion".
|
||||
Interestingly, **if a function makes its recursive call as the last step before returning**, it can be optimized by compilers or interpreters to be as space-efficient as iteration. This scenario is known as "tail recursion".
|
||||
|
||||
- **Ordinary recursion**: when a function returns to a function at a higher level, it needs to continue executing the code, so the system needs to save the context of the previous call.
|
||||
- **tail recursion**: the recursion call is the last operation before the function returns, which means that the function does not need to continue with other operations after returning to the previous level, so the system does not need to save the context of the previous function.
|
||||
- **Regular Recursion**: The function needs to perform more code after returning to the previous level, so the system needs to save the context of the previous call.
|
||||
- **Tail Recursion**: The recursive call is the last operation before the function returns, meaning no further actions are required upon returning to the previous level, so the system doesn't need to save the context of the previous level's function.
|
||||
|
||||
In the case of calculating $1 + 2 + \dots + n$, for example, we can implement tail recursion by setting the result variable `res` as a function parameter.
|
||||
For example, in calculating $1 + 2 + \dots + n$, we can make the result variable `res` a parameter of the function, thereby achieving tail recursion:
|
||||
|
||||
```src
|
||||
[file]{recursion}-[class]{}-[func]{tail_recur}
|
||||
```
|
||||
|
||||
The execution of tail recursion is shown in the figure below. Comparing normal recursion and tail recursion, the execution point of the summation operation is different.
|
||||
The execution process of tail recursion is shown in the following figure. Comparing regular recursion and tail recursion, the point of the summation operation is different.
|
||||
|
||||
- **Ordinary recursion**: the summing operation is performed during the "return" process, and the summing operation is performed again after returning from each level.
|
||||
- **Tail recursion**: the summing operation is performed in a "recursion" process, the "recursion" process simply returns in levels.
|
||||
- **Regular Recursion**: The summation operation occurs during the "return" phase, requiring another summation after each layer returns.
|
||||
- **Tail Recursion**: The summation operation occurs during the "recursion" phase, and the "return" phase only involves returning through each layer.
|
||||
|
||||
|
||||
|
||||
|
||||
!!! tip
|
||||
|
||||
Note that many compilers or interpreters do not support tail recursion optimization. For example, Python does not support tail recursion optimization by default, so even if a function is tail recursive, you may still encounter stack overflow problems.
|
||||
Note that many compilers or interpreters do not support tail recursion optimization. For example, Python does not support tail recursion optimization by default, so even if the function is in the form of tail recursion, it may still encounter stack overflow issues.
|
||||
|
||||
### Recursion Tree
|
||||
|
||||
When dealing with algorithmic problems related to divide and conquer, recursion is often more intuitive and easier to read than iteration. Take the Fibonacci sequence as an example.
|
||||
When dealing with algorithms related to "divide and conquer", recursion often offers a more intuitive approach and more readable code than iteration. Take the "Fibonacci sequence" as an example.
|
||||
|
||||
!!! question
|
||||
|
||||
Given a Fibonacci series $0, 1, 1, 2, 3, 5, 8, 13, \dots$ , find the $n$th number of the series.
|
||||
Given a Fibonacci sequence $0, 1, 1, 2, 3, 5, 8, 13, \dots$, find the $n$th number in the sequence.
|
||||
|
||||
Let the $n$th number of the Fibonacci series be $f(n)$ , which leads to two easy conclusions.
|
||||
Let the $n$th number of the Fibonacci sequence be $f(n)$, it's easy to deduce two conclusions:
|
||||
|
||||
- The first two numbers of the series are $f(1) = 0$ and $f(2) = 1$.
|
||||
- Each number in the series is the sum of the previous two numbers, i.e. $f(n) = f(n - 1) + f(n - 2)$ .
|
||||
- The first two numbers of the sequence are $f(1) = 0$ and $f(2) = 1$.
|
||||
- Each number in the sequence is the sum of the two preceding ones, that is, $f(n) = f(n - 1) + f(n - 2)$.
|
||||
|
||||
Recursion code can be written by making recursion calls according to the recursion relationship, using the first two numbers as termination conditions. Call `fib(n)` to get the $n$th number of the Fibonacci series.
|
||||
Using the recursive relation, and considering the first two numbers as termination conditions, we can write the recursive code. Calling `fib(n)` will yield the $n$th number of the Fibonacci sequence:
|
||||
|
||||
```src
|
||||
[file]{recursion}-[class]{}-[func]{fib}
|
||||
```
|
||||
|
||||
Looking at the above code, we have recursively called two functions within a function, **this means that from one call, two call branches are created**. As shown in the figure below, this recursion will result in a recursion tree with the number of levels $n$.
|
||||
Observing the above code, we see that it recursively calls two functions within itself, **meaning that one call generates two branching calls**. As illustrated below, this continuous recursive calling eventually creates a "recursion tree" with a depth of $n$.
|
||||
|
||||
|
||||
|
||||
|
||||
Essentially, recursion embodies the paradigm of "breaking down a problem into smaller sub-problems", and this divide and conquer strategy is essential.
|
||||
Fundamentally, recursion embodies the paradigm of "breaking down a problem into smaller sub-problems." This divide-and-conquer strategy is crucial.
|
||||
|
||||
- From an algorithmic point of view, many important algorithmic strategies such as searching, sorting algorithm, backtracking, divide and conquer, dynamic programming, etc. directly or indirectly apply this way of thinking.
|
||||
- From a data structure point of view, recursion is naturally suited to problems related to linked lists, trees and graphs because they are well suited to be analyzed with the idea of partitioning.
|
||||
- From an algorithmic perspective, many important strategies like searching, sorting, backtracking, divide-and-conquer, and dynamic programming directly or indirectly use this way of thinking.
|
||||
- From a data structure perspective, recursion is naturally suited for dealing with linked lists, trees, and graphs, as they are well suited for analysis using the divide-and-conquer approach.
|
||||
|
||||
## Compare The Two
|
||||
## Comparison
|
||||
|
||||
To summarize the above, as shown in the table below, iteration and recursion differ in implementation, performance and applicability.
|
||||
Summarizing the above content, the following table shows the differences between iteration and recursion in terms of implementation, performance, and applicability.
|
||||
|
||||
<p align="center"> Table <id> Comparison of iteration and recursion features </p>
|
||||
<p align="center"> Table: Comparison of Iteration and Recursion Characteristics </p>
|
||||
|
||||
| | iteration | recursion |
|
||||
| ------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| implementation | circular structure | function call itself |
|
||||
| time-efficient | typically efficient, no function call overhead | overhead on every function call |
|
||||
| Memory Usage | Usually uses a fixed size of memory space | Cumulative function calls may use a lot of stack frame space |
|
||||
| Applicable Problems | For simple cyclic tasks, code is intuitive and readable | For sub-problem decomposition, such as trees, graphs, divide and conquer, backtracking, etc., the code structure is concise and clear |
|
||||
| | Iteration | Recursion |
|
||||
| ----------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Approach | Loop structure | Function calls itself |
|
||||
| Time Efficiency | Generally higher efficiency, no function call overhead | Each function call generates overhead |
|
||||
| Memory Usage | Typically uses a fixed size of memory space | Accumulative function calls can use a substantial amount of stack frame space |
|
||||
| Suitable Problems | Suitable for simple loop tasks, intuitive and readable code | Suitable for problem decomposition, like trees, graphs, divide-and-conquer, backtracking, etc., concise and clear code structure |
|
||||
|
||||
!!! tip
|
||||
|
||||
If you find the following solutions difficult to understand, you can review them after reading the "Stack" chapter.
|
||||
If you find the following content difficult to understand, consider revisiting it after reading the "Stack" chapter.
|
||||
|
||||
So what is the intrinsic connection between iteration and recursion? In the case of the recursive function described above, the summing operation takes place in the "return" phase of the recursion. This means that the function that is initially called is actually the last to complete its summing operation, **This mechanism works in the same way as the stack's "first in, last out" principle**.
|
||||
So, what is the intrinsic connection between iteration and recursion? Taking the above recursive function as an example, the summation operation occurs during the recursion's "return" phase. This means that the initially called function is actually the last to complete its summation operation, **mirroring the "last in, first out" principle of a stack**.
|
||||
|
||||
In fact, recursion terms like "call stack" and "stack frame space" already imply a close relationship between recursion and the stack.
|
||||
In fact, recursive terms like "call stack" and "stack frame space" hint at the close relationship between recursion and stacks.
|
||||
|
||||
1. **Recursive**: When a function is called, the system allocates a new stack frame on the "call stack" for the function, which is used to store the function's local variables, parameters, return address, and other data.
|
||||
2. **Return to**: When a function completes execution and returns, the corresponding stack frame is removed from the "call stack", restoring the function's previous execution environment.
|
||||
1. **Recursion**: When a function is called, the system allocates a new stack frame on the "call stack" for that function, storing local variables, parameters, return addresses, and other data.
|
||||
2. **Return**: When a function completes execution and returns, the corresponding stack frame is removed from the "call stack," restoring the execution environment of the previous function.
|
||||
|
||||
Thus, **we can use an explicit stack to model the behavior of the call stack**, thus transforming recursion into an iteration form:
|
||||
Therefore, **we can use an explicit stack to simulate the behavior of the call stack**, thus transforming recursion into an iterative form:
|
||||
|
||||
```src
|
||||
[file]{recursion}-[class]{}-[func]{for_loop_recur}
|
||||
```
|
||||
|
||||
Observing the code above, it becomes more complex when recursion is converted to iteration. Although iteration and recursion can be converted to each other in many cases, it is not always worth doing so for two reasons.
|
||||
Observing the above code, when recursion is transformed into iteration, the code becomes more complex. Although iteration and recursion can often be transformed into each other, it's not always advisable to do so for two reasons:
|
||||
|
||||
- The transformed code may be more difficult to understand and less readable.
|
||||
- For some complex problems, simulating the behavior of the system call stack can be very difficult.
|
||||
- The transformed code may become harder to understand and less readable.
|
||||
- For some complex problems, simulating the behavior of the system's call stack can be quite challenging.
|
||||
|
||||
In short, **the choice of iteration or recursion depends on the nature of the particular problem**. In programming practice, it is crucial to weigh the advantages and disadvantages of both and choose the appropriate method based on the context.
|
||||
In summary, **choosing between iteration and recursion depends on the nature of the specific problem**. In programming practice, weighing the pros and cons of each and choosing the appropriate method for the situation is essential.
|
||||
|
||||
@ -1,49 +1,48 @@
|
||||
# Evaluation Of Algorithm Efficiency
|
||||
# Algorithm Efficiency Assessment
|
||||
|
||||
In algorithm design, we aim to achieve two goals in succession:
|
||||
In algorithm design, we pursue the following two objectives in sequence.
|
||||
|
||||
1. **Finding a Solution to the Problem**: The algorithm needs to reliably find the correct solution within the specified input range.
|
||||
2. **Seeking the Optimal Solution**: There may be multiple ways to solve the same problem, and our goal is to find the most efficient algorithm possible.
|
||||
1. **Finding a Solution to the Problem**: The algorithm should reliably find the correct solution within the stipulated range of inputs.
|
||||
2. **Seeking the Optimal Solution**: For the same problem, multiple solutions might exist, and we aim to find the most efficient algorithm possible.
|
||||
|
||||
In other words, once the ability to solve the problem is established, the efficiency of the algorithm emerges as the main benchmark for assessing its quality, which includes the following two aspects.
|
||||
In other words, under the premise of being able to solve the problem, algorithm efficiency has become the main criterion for evaluating the merits of an algorithm, which includes the following two dimensions.
|
||||
|
||||
- **Time Efficiency**: The speed at which an algorithm runs.
|
||||
- **Space Efficiency**: The amount of memory space the algorithm consumes.
|
||||
- **Space Efficiency**: The size of the memory space occupied by an algorithm.
|
||||
|
||||
In short, our goal is to design data structures and algorithms that are both "fast and economical". Effectively evaluating algorithm efficiency is crucial, as it allows for the comparison of different algorithms and guides the design and optimization process.
|
||||
In short, **our goal is to design data structures and algorithms that are both fast and memory-efficient**. Effectively assessing algorithm efficiency is crucial because only then can we compare various algorithms and guide the process of algorithm design and optimization.
|
||||
|
||||
There are mainly two approaches for assessing efficiency: practical testing and theoretical estimation.
|
||||
There are mainly two methods of efficiency assessment: actual testing and theoretical estimation.
|
||||
|
||||
## Practical Testing
|
||||
## Actual Testing
|
||||
|
||||
Let's consider a scenario where we have two algorithms, `A` and `B`, both capable of solving the same problem. To compare their efficiency, the most direct method is to use a computer to run both algorithms while monitoring and recording their execution time and memory usage. This approach provides a realistic assessment of their performance, but it also has significant limitations.
|
||||
Suppose we have algorithms `A` and `B`, both capable of solving the same problem, and we need to compare their efficiencies. The most direct method is to use a computer to run these two algorithms and monitor and record their runtime and memory usage. This assessment method reflects the actual situation but has significant limitations.
|
||||
|
||||
On one hand, it's challenging to eliminate the interference of the test environment. Hardware configurations can significantly affect the performance of algorithms. For instance, on one computer, Algorithm `A` might run faster than Algorithm `B`, but the results could be the opposite on another computer with different specifications. This means we would need to conduct tests on a variety of machines and calculate an average efficiency, which is impractical.
|
||||
On one hand, **it's difficult to eliminate interference from the testing environment**. Hardware configurations can affect algorithm performance. For example, algorithm `A` might run faster than `B` on one computer, but the opposite result may occur on another computer with different configurations. This means we would need to test on a variety of machines to calculate average efficiency, which is impractical.
|
||||
|
||||
Furthermore, conducting comprehensive tests is resource-intensive. The efficiency of algorithms can vary with different volumes of input data. For example, with smaller data sets, Algorithm A might run faster than Algorithm B; however, this could change with larger data sets. Therefore, to reach a convincing conclusion, it's necessary to test a range of data sizes, which requires excessive computational resources.
|
||||
On the other hand, **conducting a full test is very resource-intensive**. As the volume of input data changes, the efficiency of the algorithms may vary. For example, with smaller data volumes, algorithm `A` might run faster than `B`, but the opposite might be true with larger data volumes. Therefore, to draw convincing conclusions, we need to test a wide range of input data sizes, which requires significant computational resources.
|
||||
|
||||
## Theoretical Estimation
|
||||
|
||||
Given the significant limitations of practical testing, we can consider assessing algorithm efficiency solely through calculations. This method of estimation is known as 'asymptotic complexity analysis,' often simply referred to as 'complexity analysis.
|
||||
Due to the significant limitations of actual testing, we can consider evaluating algorithm efficiency solely through calculations. This estimation method is known as "asymptotic complexity analysis," or simply "complexity analysis."
|
||||
|
||||
Complexity analysis illustrates the relationship between the time (and space) resources required by an algorithm and the size of its input data. **It describes the growing trend in the time and space required for the execution of an algorithm as the size of the input data increases**. This definition might sound a bit complex, so let's break it down into three key points for easier understanding.
|
||||
Complexity analysis reflects the relationship between the time and space resources required for algorithm execution and the size of the input data. **It describes the trend of growth in the time and space required by the algorithm as the size of the input data increases**. This definition might sound complex, but we can break it down into three key points to understand it better.
|
||||
|
||||
- In complexity analysis, 'time and space' directly relate to 'time complexity' and 'space complexity,' respectively.
|
||||
- The statement "as the size of the input data increases" highlights that complexity analysis examines the interplay between the size of the input data and the algorithm's efficiency.
|
||||
- Lastly, the phrase "the growing trend in time and space required" emphasizes that the focus of complexity analysis is not on the specific values of running time or space occupied, but on the rate at which these requirements increase with larger input sizes.
|
||||
- "Time and space resources" correspond to "time complexity" and "space complexity," respectively.
|
||||
- "As the size of input data increases" means that complexity reflects the relationship between algorithm efficiency and the volume of input data.
|
||||
- "The trend of growth in time and space" indicates that complexity analysis focuses not on the specific values of runtime or space occupied but on the "rate" at which time or space grows.
|
||||
|
||||
**Complexity analysis overcomes the drawbacks of practical testing methods in two key ways:**.
|
||||
**Complexity analysis overcomes the disadvantages of actual testing methods**, reflected in the following aspects:
|
||||
|
||||
- It is independent of the testing environment, meaning the analysis results are applicable across all operating platforms.
|
||||
- It effectively demonstrates the efficiency of algorithms with varying data volumes, particularly highlighting performance in large-scale data scenarios.
|
||||
- It is independent of the testing environment and applicable to all operating platforms.
|
||||
- It can reflect algorithm efficiency under different data volumes, especially in the performance of algorithms with large data volumes.
|
||||
|
||||
!!! tip
|
||||
|
||||
If you're still finding the concept of complexity confusing, don't worry. We will cover it in more detail in the subsequent chapters.
|
||||
If you're still confused about the concept of complexity, don't worry. We will introduce it in detail in subsequent chapters.
|
||||
|
||||
Complexity analysis provides us with a 'ruler' for evaluating the efficiency of algorithms, enabling us to measure the time and space resources required to execute a given algorithm and to compare the efficiency of different algorithms.
|
||||
Complexity analysis provides us with a "ruler" to measure the time and space resources needed to execute an algorithm and compare the efficiency between different algorithms.
|
||||
|
||||
Complexity is a mathematical concept that might seem abstract and somewhat challenging for beginners. From this perspective, introducing complexity analysis at the very beginning may not be the most suitable approach. However, when discussing the characteristics of a particular data structure or algorithm, analyzing its operational speed and space usage is often inevitable.
|
||||
|
||||
Therefore, it is recommended that before diving deeply into data structures and algorithms, **one should first gain a basic understanding of complexity analysis. This foundational knowledge will facilitate the complexity analysis of simple algorithms.**
|
||||
Complexity is a mathematical concept and may be abstract and challenging for beginners. From this perspective, complexity analysis might not be the best content to introduce first. However, when discussing the characteristics of a particular data structure or algorithm, it's hard to avoid analyzing its speed and space usage.
|
||||
|
||||
In summary, it's recommended that you establish a preliminary understanding of complexity analysis before diving deep into data structures and algorithms, **so that you can carry out simple complexity analyses of algorithms**.
|
||||
|
||||
@ -1,49 +1,49 @@
|
||||
# Summary
|
||||
|
||||
### Highlights
|
||||
### Key Review
|
||||
|
||||
**Evaluation of Algorithm Efficiency**
|
||||
**Algorithm Efficiency Assessment**
|
||||
|
||||
- Time and space efficiency are the two leading evaluation indicators to measure an algorithm.
|
||||
- We can evaluate the efficiency of an algorithm through real-world testing. Still, it isn't easy to eliminate the side effects from the testing environment, and it consumes a lot of computational resources.
|
||||
- Complexity analysis overcomes the drawbacks of real-world testing. The analysis results can apply to all operating platforms and reveal the algorithm's efficiency under variant data scales.
|
||||
- Time efficiency and space efficiency are the two main criteria for assessing the merits of an algorithm.
|
||||
- We can assess algorithm efficiency through actual testing, but it's challenging to eliminate the influence of the test environment, and it consumes substantial computational resources.
|
||||
- Complexity analysis can overcome the disadvantages of actual testing. Its results are applicable across all operating platforms and can reveal the efficiency of algorithms at different data scales.
|
||||
|
||||
**Time Complexity**
|
||||
|
||||
- Time complexity is used to measure the trend of algorithm running time as the data size grows., which can effectively evaluate the algorithm's efficiency. However, it may fail in some cases, such as when the input volume is small or the time complexities are similar, making it difficult to precisely compare the efficiency of algorithms.
|
||||
- The worst time complexity is denoted by big $O$ notation, which corresponds to the asymptotic upper bound of the function, reflecting the growth rate in the number of operations $T(n)$ as $n$ tends to positive infinity.
|
||||
- The estimation of time complexity involves two steps: first, counting the number of operations, and then determining the asymptotic upper bound.
|
||||
- Common time complexities, from lowest to highest, are $O(1)$, $O(\log n)$, $O(n)$, $O(n \log n)$, $O(n^2)$, $O(2^n)$, and $O(n!)$.
|
||||
- The time complexity of certain algorithms is not fixed and depends on the distribution of the input data. The time complexity can be categorized into worst-case, best-case, and average. The best-case time complexity is rarely used because the input data must meet strict conditions to achieve the best-case.
|
||||
- The average time complexity reflects the efficiency of an algorithm with random data inputs, which is closest to the performance of algorithms in real-world scenarios. Calculating the average time complexity requires statistical analysis of input data and a synthesized mathematical expectation.
|
||||
- Time complexity measures the trend of an algorithm's running time with the increase in data volume, effectively assessing algorithm efficiency. However, it can fail in certain cases, such as with small input data volumes or when time complexities are the same, making it challenging to precisely compare the efficiency of algorithms.
|
||||
- Worst-case time complexity is denoted using big O notation, representing the asymptotic upper bound, reflecting the growth level of the number of operations $T(n)$ as $n$ approaches infinity.
|
||||
- Calculating time complexity involves two steps: first counting the number of operations, then determining the asymptotic upper bound.
|
||||
- Common time complexities, arranged from low to high, include $O(1)$, $O(\log n)$, $O(n)$, $O(n \log n)$, $O(n^2)$, $O(2^n)$, and $O(n!)$, among others.
|
||||
- The time complexity of some algorithms is not fixed and depends on the distribution of input data. Time complexities are divided into worst, best, and average cases. The best case is rarely used because input data generally needs to meet strict conditions to achieve the best case.
|
||||
- Average time complexity reflects the efficiency of an algorithm under random data inputs, closely resembling the algorithm's performance in actual applications. Calculating average time complexity requires accounting for the distribution of input data and the subsequent mathematical expectation.
|
||||
|
||||
**Space Complexity**
|
||||
|
||||
- Space complexity serves a similar purpose to time complexity and is used to measure the trend of space occupied by an algorithm as the data volume increases.
|
||||
- The memory space associated with the operation of an algorithm can be categorized into input space, temporary space, and output space. Normally, the input space is not considered when determining space complexity. The temporary space can be classified into instruction space, data space, and stack frame space, and the stack frame space usually only affects the space complexity for recursion functions.
|
||||
- We mainly focus on the worst-case space complexity, which refers to the measurement of an algorithm's space usage when given the worst-case input data and during the worst-case execution scenario.
|
||||
- Common space complexities are $O(1)$, $O(\log n)$, $O(n)$, $O(n^2)$ and $O(2^n)$ from lowest to highest.
|
||||
- Space complexity, similar to time complexity, measures the trend of memory space occupied by an algorithm with the increase in data volume.
|
||||
- The relevant memory space used during the algorithm's execution can be divided into input space, temporary space, and output space. Generally, input space is not included in space complexity calculations. Temporary space can be divided into temporary data, stack frame space, and instruction space, where stack frame space usually affects space complexity only in recursive functions.
|
||||
- We usually focus only on the worst-case space complexity, which means calculating the space complexity of the algorithm under the worst input data and at the worst moment of operation.
|
||||
- Common space complexities, arranged from low to high, include $O(1)$, $O(\log n)$, $O(n)$, $O(n^2)$, and $O(2^n)$, among others.
|
||||
|
||||
### Q & A
|
||||
|
||||
!!! question "Is the space complexity of tail recursion $O(1)$?"
|
||||
|
||||
Theoretically, the space complexity of a tail recursion function can be optimized to $O(1)$. However, most programming languages (e.g., Java, Python, C++, Go, C#, etc.) do not support auto-optimization for tail recursion, so the space complexity is usually considered as $O(n)$.
|
||||
Theoretically, the space complexity of a tail-recursive function can be optimized to $O(1)$. However, most programming languages (such as Java, Python, C++, Go, C#) do not support automatic optimization of tail recursion, so it's generally considered to have a space complexity of $O(n)$.
|
||||
|
||||
!!! question "What is the difference between the terms function and method?"
|
||||
!!! question "What is the difference between the terms 'function' and 'method'?"
|
||||
|
||||
A *function* can be executed independently, and all arguments are passed explicitly. A *method* is associated with an object and is implicitly passed to the object that calls it, allowing it to operate on the data contained within an instance of a class.
|
||||
A "function" can be executed independently, with all parameters passed explicitly. A "method" is associated with an object and is implicitly passed to the object calling it, able to operate on the data contained within an instance of a class.
|
||||
|
||||
Let's illustrate with a few common programming languages.
|
||||
Here are some examples from common programming languages:
|
||||
|
||||
- C is a procedural programming language without object-oriented concepts, so it has only functions. However, we can simulate object-oriented programming by creating structures (struct), and the functions associated with structures are equivalent to methods in other languages.
|
||||
- Java and C# are object-oriented programming languages, and blocks of code (methods) are typically part of a class. Static methods behave like a function because it is bound to the class and cannot access specific instance variables.
|
||||
- Both C++ and Python support both procedural programming (functions) and object-oriented programming (methods).
|
||||
- C is a procedural programming language without object-oriented concepts, so it only has functions. However, we can simulate object-oriented programming by creating structures (struct), and functions associated with these structures are equivalent to methods in other programming languages.
|
||||
- Java and C# are object-oriented programming languages where code blocks (methods) are typically part of a class. Static methods behave like functions because they are bound to the class and cannot access specific instance variables.
|
||||
- C++ and Python support both procedural programming (functions) and object-oriented programming (methods).
|
||||
|
||||
!!! question "Does the figure "Common Types of Space Complexity" reflect the absolute size of the occupied space?"
|
||||
!!! question "Does the 'Common Types of Space Complexity' figure reflect the absolute size of occupied space?"
|
||||
|
||||
No, that figure shows the space complexity, which reflects the growth trend, not the absolute size of the space occupied.
|
||||
|
||||
For example, if you take $n = 8$ , the values of each curve do not align with the function because each curve contains a constant term used to compress the range of values to a visually comfortable range.
|
||||
No, the figure shows space complexities, which reflect growth trends, not the absolute size of the occupied space.
|
||||
|
||||
In practice, since we usually don't know each method's "constant term" complexity, it is generally impossible to choose the optimal solution for $n = 8$ based on complexity alone. But it's easier to choose for $n = 8^5$ as the growth trend is already dominant.
|
||||
If you take $n = 8$, you might find that the values of each curve don't correspond to their functions. This is because each curve includes a constant term, intended to compress the value range into a visually comfortable range.
|
||||
|
||||
In practice, since we usually don't know the "constant term" complexity of each method, it's generally not possible to choose the best solution for $n = 8$ based solely on complexity. However, for $n = 8^5$, it's much easier to choose, as the growth trend becomes dominant.
|
||||
|
||||
Loading…
Reference in new issue