首页 > 代码库 > 保存最后N个元素

保存最后N个元素

cookbook系列

问题:对要搜索的值的最后几项做个有限的历史记录。

方案:

 1 #coding=utf-8
 2 from collections import deque
 3 
 4 def search(lines, pattern, history=5):
 5     previous_lines = deque(maxlen=history)  #deque双端队列
 6     for line in lines:
 7         if pattern in line:
 8             yield line, previous_lines      #生成器,双端队列保存5个,意味着在找到pattern之前会记录5行数据,pattern就在line里面
 9         previous_lines.append(line)
10 
11 # Example use on a file
12 if __name__ == __main__:
13     with open(somefile.txt) as f:
14         for line, prevlines in search(f, python, 5):
15             for pline in prevlines:
16                 print pline,  #先输出队列里的5个
17             print line,     #在输出pattern的line
18             print(-*20)

案例文件:somefile.txt

技术分享
=== Keeping the Last N Items

==== Problem

You want to keep a limited history of the last few items seen
during iteration or during some other kind of processing.

==== Solution

Keeping a limited history is a perfect use for a `collections.deque`.
For example, the following code performs a simple text match on a
sequence of lines and prints the matching line along with the previous
N lines of context when found:

[source,python]
----
from collections import deque

def search(lines, pattern, history=5):
    previous_lines = deque(maxlen=history)
    for line in lines:
        if pattern in line:
            for pline in previous_lines:
                print(lline, end=‘‘)
            print(line, end=‘‘)
            print()
        previous_lines.append(line)

# Example use on a file
if __name__ == __main__:
    with open(somefile.txt) as f:
         search(f, python, 5)
----

==== Discussion

Using `deque(maxlen=N)` creates a fixed size queue.  When new items
are added and the queue is full, the oldest item is automatically
removed.   For example:

[source,pycon]
----
>>> q = deque(maxlen=3)
>>> q.append(1)
>>> q.append(2)
>>> q.append(3)
>>> q
deque([1, 2, 3], maxlen=3)
>>> q.append(4)
>>> q
deque([2, 3, 4], maxlen=3)
>>> q.append(5)
>>> q
deque([3, 4, 5], maxlen=3)
----

Although you could manually perform such operations on a list (e.g.,
appending, deleting, etc.), the queue solution is far more elegant and
runs a lot faster.

More generally, a `deque` can be used whenever you need a simple queue
structure.  If you dont give it a maximum size, you get an unbounded
queue that lets you append and pop items on either end.  For example:

[source,pycon]
----
>>> q = deque()
>>> q.append(1)
>>> q.append(2)
>>> q.append(3)
>>> q
deque([1, 2, 3])
>>> q.appendleft(4)
>>> q
deque([4, 1, 2, 3])
>>> q.pop()
3
>>> q
deque([4, 1, 2])
>>> q.popleft()
4
----

Adding or popping items from either end of a queue has O(1)
complexity.   This is unlike a list where inserting or removing
items from the front of the list is O(N).    
View Code

运行结果:

技术分享
H:\Python27_64\python.exe H:/myfile/python-cookbook-master/src/1/keeping_the_last_n_items/example.py
Keeping a limited history is a perfect use for a `collections.deque`.
For example, the following code performs a simple text match on a
sequence of lines and prints the matching line along with the previous
N lines of context when found:

[source,python]
--------------------
        previous_lines.append(line)

# Example use on a file
if __name__ == __main__:
    with open(somefile.txt) as f:
         search(f, python, 5)
--------------------

进程已结束,退出代码0
View Code

讨论:

第5行双端队列deque的用法

固定长度队列:
>>> q = deque(maxlen=3) >>> q.append(1) >>> q.append(2) >>> q.append(3) >>> q deque([1, 2, 3], maxlen=3) >>> q.append(4) >>> q deque([2, 3, 4], maxlen=3) >>> q.append(5) >>> q deque([3, 4, 5], maxlen=3) 双端队列: >>> q = deque() >>> q.append(1) >>> q.append(2) >>> q.append(3) >>> q deque([1, 2, 3]) >>> q.appendleft(4) >>> q deque([4, 1, 2, 3]) >>> q.pop() 3 >>> q deque([4, 1, 2]) >>> q.popleft() 4
两者皆有:设置好固定长度时,左右两端均可添加和删除

技术分享

 

使用双端队列要比列表 方便、简洁!

保存最后N个元素