python 多进程并行编程 ProcessPoolExecutor的实现

Python  /  管理员 发布于 7年前   165

使用 ProcessPoolExecutor

from concurrent.futures import ProcessPoolExecutor, as_completedimport random

斐波那契数列

当 n 大于 30 时抛出异常

def fib(n):  if n > 30:    raise Exception('can not > 30, now %s' % n)  if n <= 2:    return 1  return fib(n-1) + fib(n-2)

准备数组

nums = [random.randint(0, 33) for _ in range(0, 10)]'''[13, 17, 0, 22, 19, 33, 7, 12, 8, 16]'''

方案一：submit

submit 输出结果按照子进程执行结束的先后顺序，不可控

 with ProcessPoolExecutor(max_workers=3) as executor:    futures = {executor.submit(fib, n):n for n in nums}    for f in as_completed(futures):      try:        print('fib(%s) result is %s.' % (futures[f], f.result()))      except Exception as e:        print(e)'''fib(13) result is 233.fib(17) result is 1597.fib(0) result is 1.fib(22) result is 17711.fib(19) result is 4181.can not > 30, now 33fib(7) result is 13.fib(12) result is 144.fib(8) result is 21.fib(16) result is 987.'''

等价写法：

 with ProcessPoolExecutor(max_workers=3) as executor:    futures = {}    for n in nums:      job = executor.submit(fib, n)      futures[job] = n    for job in as_completed(futures):      try:        re = job.result()        n = futures[job]        print('fib(%s) result is %s.' % (n, re))      except Exception as e:        print(e)'''fib(13) result is 233.fib(17) result is 1597.fib(0) result is 1.fib(22) result is 17711.can not > 30, now 33fib(7) result is 13.fib(19) result is 4181.fib(8) result is 21.fib(12) result is 144.fib(16) result is 987.'''

方案二：map

map 输出结果按照输入数组的顺序

缺点：某一子进程异常会导致整体中断

 with ProcessPoolExecutor(max_workers=3) as executor:    try:      results = executor.map(fib, nums)      for num, result in zip(nums, results):        print('fib(%s) result is %s.' % (num, result))    except Exception as e:      print(e)'''fib(13) result is 233.fib(17) result is 1597.fib(0) result is 1.fib(22) result is 17711.fib(19) result is 4181.can not > 30, now 33'''

以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持。