首页 > 代码库 > Python字节码与解释器学习

Python字节码与解释器学习

2024-07-11 15:55:07   222人阅读

参考：http://blog.jobbole.com/55327/

http://blog.jobbole.com/56300/

http://blog.jobbole.com/56761/

 

1. 在交互式命令行中执行命令的内部过程

当你敲下return键的时候，python完成了以下四步：词法分析、句法分析、编译、解释。词法分析的工作就是将你刚才输入的那行代码分解为一些符号token（译者注：包括标示符，关键字，数字, 操作符等）。句法分析程序再接收这些符号，并用一种结构来展现它们之间的关系（在这种情况下使用的抽象语法树）。然后编译器接收这棵抽象语法树，并将它转化为一个（或多个）代码对象。最后，解释器逐个接收这些代码对象，并执行它们所代表的代码。

 

每一行我们输入的命令，都要经过上面的四个步骤，才能够被执行。

 

2. 函数对象

对象是面向对象理论中的基本元素，在一些动态或者解释性语言中，函数也可以看作是一种对象，比如在JavaScript，以及功能性编程语言Haskell/Ocaml中，函数都是一种特殊的对象。

 

函数是对象，就意味着函数可以像对象一样被执行各种操作，比如分配，释放，复制，赋值......

“函数是最好的对象”说明函数是一种对象。它就如同一个列表或者举个例子来说 ：MyObject 就是一个对象。既然 foo 是一个对象，那么我们就能在不调用它的情况下使用它（也就是说，foo 和 foo() 是大相径庭的）。我们能够将 foo 当作一个参数传递给另一个函数或者赋值给一个新函数名（ other_function = foo ）。有了如此棒的函数，一切皆为可能！

 

另外，函数作为对象出现的时候，就是和函数调用有区别的，函数调用是一个动态的过程；而函数作为一个对象，是一个静态的实体概念，意思是你可以对这个对象施予一些操作，这与这个对象的类型有关，或者以面向对象的思想来说，你可以执行这个对象提供的各种接口操作（函数）。

 

既然是对象，那么函数对象有哪些成员呢?

>>> dir<built-in function dir>>>> dir(dir)[‘__call__‘, ‘__class__‘, ‘__cmp__‘, ‘__delattr__‘, ‘__doc__‘, ‘__eq__‘, ‘__format__‘, ‘__ge__‘, ‘__getattribute__‘, ‘__gt__‘, ‘__hash__‘, ‘__init__‘, ‘__le__‘, ‘__lt__‘, ‘__module__‘, ‘__name__‘, ‘__ne__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__self__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘]>>> dir(dir.func_code)Traceback (most recent call last):  File "<pyshell#2>", line 1, in <module>    dir(dir.func_code)AttributeError: ‘builtin_function_or_method‘ object has no attribute ‘func_code‘>>> def foo(a):	x = 3	return x + a>>> foo<function foo at 0x0000000002E8F128>>>> dir(foo)[‘__call__‘, ‘__class__‘, ‘__closure__‘, ‘__code__‘, ‘__defaults__‘, ‘__delattr__‘, ‘__dict__‘, ‘__doc__‘, ‘__format__‘, ‘__get__‘, ‘__getattribute__‘, ‘__globals__‘, ‘__hash__‘, ‘__init__‘, ‘__module__‘, ‘__name__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘func_closure‘, ‘func_code‘, ‘func_defaults‘, ‘func_dict‘, ‘func_doc‘, ‘func_globals‘, ‘func_name‘]>>>

其中，内置函数dir的功能描述如下：

dir([object])

Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object.

If the object has a method named __dir__(), this method will be called and must return the list of attributes. This allows objects that implement a custom __getattr__() or __getattribute__() function to customize the way dir() reports their attributes.

If the object does not provide __dir__(), the function tries its best to gather information from the object’s __dict__ attribute, if defined, and from its type object. The resulting list is not necessarily complete, and may be inaccurate when the object has a custom __getattr__().

The default dir() mechanism behaves differently with different types of objects, as it attempts to produce the most relevant, rather than complete, information:

  • If the object is a module object, the list contains the names of the module’s attributes.
  • If the object is a type or class object, the list contains the names of its attributes, and recursively of the attributes of its bases.
  • Otherwise, the list contains the object’s attributes’ names, the names of its class’s attributes, and recursively of the attributes of its class’s base classes.

The resulting list is sorted alphabetically.

　　

除此之外，help内置函数也很重要，可以查看内置函数的帮助内容。

 

首先，查看当前Python程序加载了哪些模块

>>> for i in sys.modules.keys():...     print "%20s:\t%s\n" % (i, sys.modules[i])...     print "*"*100

 

            copy_reg:	<module ‘copy_reg‘ from ‘/usr/lib/python2.7/copy_reg.pyc‘>****************************************************************************************************         sre_compile:	<module ‘sre_compile‘ from ‘/usr/lib/python2.7/sre_compile.pyc‘>****************************************************************************************************                _sre:	<module ‘_sre‘ (built-in)>****************************************************************************************************           encodings:	<module ‘encodings‘ from ‘/usr/lib/python2.7/encodings/__init__.pyc‘>****************************************************************************************************                site:	<module ‘site‘ from ‘/usr/lib/python2.7/site.pyc‘>****************************************************************************************************         __builtin__:	<module ‘__builtin__‘ (built-in)>****************************************************************************************************           sysconfig:	<module ‘sysconfig‘ from ‘/usr/lib/python2.7/sysconfig.pyc‘>****************************************************************************************************            __main__:	<module ‘__main__‘ (built-in)>**************************************************************************************************** encodings.encodings:	None****************************************************************************************************                 abc:	<module ‘abc‘ from ‘/usr/lib/python2.7/abc.pyc‘>****************************************************************************************************           posixpath:	<module ‘posixpath‘ from ‘/usr/lib/python2.7/posixpath.pyc‘>****************************************************************************************************         _weakrefset:	<module ‘_weakrefset‘ from ‘/usr/lib/python2.7/_weakrefset.pyc‘>****************************************************************************************************               errno:	<module ‘errno‘ (built-in)>****************************************************************************************************    encodings.codecs:	None****************************************************************************************************       sre_constants:	<module ‘sre_constants‘ from ‘/usr/lib/python2.7/sre_constants.pyc‘>****************************************************************************************************                  re:	<module ‘re‘ from ‘/usr/lib/python2.7/re.pyc‘>****************************************************************************************************             _abcoll:	<module ‘_abcoll‘ from ‘/usr/lib/python2.7/_abcoll.pyc‘>****************************************************************************************************               types:	<module ‘types‘ from ‘/usr/lib/python2.7/types.pyc‘>****************************************************************************************************             _codecs:	<module ‘_codecs‘ (built-in)>****************************************************************************************************encodings.__builtin__:	None****************************************************************************************************           _warnings:	<module ‘_warnings‘ (built-in)>****************************************************************************************************         genericpath:	<module ‘genericpath‘ from ‘/usr/lib/python2.7/genericpath.pyc‘>****************************************************************************************************                stat:	<module ‘stat‘ from ‘/usr/lib/python2.7/stat.pyc‘>****************************************************************************************************           zipimport:	<module ‘zipimport‘ (built-in)>****************************************************************************************************      _sysconfigdata:	<module ‘_sysconfigdata‘ from ‘/usr/lib/python2.7/_sysconfigdata.pyc‘>****************************************************************************************************            warnings:	<module ‘warnings‘ from ‘/usr/lib/python2.7/warnings.pyc‘>****************************************************************************************************            UserDict:	<module ‘UserDict‘ from ‘/usr/lib/python2.7/UserDict.pyc‘>****************************************************************************************************     encodings.utf_8:	<module ‘encodings.utf_8‘ from ‘/usr/lib/python2.7/encodings/utf_8.pyc‘>****************************************************************************************************                 sys:	<module ‘sys‘ (built-in)>****************************************************************************************************              codecs:	<module ‘codecs‘ from ‘/usr/lib/python2.7/codecs.pyc‘>****************************************************************************************************            readline:	<module ‘readline‘ from ‘/usr/lib/python2.7/lib-dynload/readline.i386-linux-gnu.so‘>****************************************************************************************************   _sysconfigdata_nd:	<module ‘_sysconfigdata_nd‘ from ‘/usr/lib/python2.7/plat-i386-linux-gnu/_sysconfigdata_nd.pyc‘>****************************************************************************************************             os.path:	<module ‘posixpath‘ from ‘/usr/lib/python2.7/posixpath.pyc‘>****************************************************************************************************       sitecustomize:	<module ‘sitecustomize‘ from ‘/usr/lib/python2.7/sitecustomize.pyc‘>****************************************************************************************************              signal:	<module ‘signal‘ (built-in)>****************************************************************************************************           traceback:	<module ‘traceback‘ from ‘/usr/lib/python2.7/traceback.pyc‘>****************************************************************************************************           linecache:	<module ‘linecache‘ from ‘/usr/lib/python2.7/linecache.pyc‘>****************************************************************************************************               posix:	<module ‘posix‘ (built-in)>****************************************************************************************************   encodings.aliases:	<module ‘encodings.aliases‘ from ‘/usr/lib/python2.7/encodings/aliases.pyc‘>****************************************************************************************************          exceptions:	<module ‘exceptions‘ (built-in)>****************************************************************************************************           sre_parse:	<module ‘sre_parse‘ from ‘/usr/lib/python2.7/sre_parse.pyc‘>****************************************************************************************************                  os:	<module ‘os‘ from ‘/usr/lib/python2.7/os.pyc‘>****************************************************************************************************            _weakref:	<module ‘_weakref‘ (built-in)>****************************************************************************************************

　　

可以通过下面代码查看__builtin__模块中的成员

>>> num = 0>>> for i in dir(sys.modules["__builtin__"]):...     print "%20s\t" % i,...     num += 1...     if num == 5:...             print ""...             num = 0...      ArithmeticError	      AssertionError	      AttributeError	       BaseException	         BufferError	        BytesWarning	  DeprecationWarning	            EOFError	            Ellipsis	    EnvironmentError	           Exception	               False	  FloatingPointError	       FutureWarning	       GeneratorExit	             IOError	         ImportError	       ImportWarning	    IndentationError	          IndexError	            KeyError	   KeyboardInterrupt	         LookupError	         MemoryError	           NameError	                None	      NotImplemented	 NotImplementedError	             OSError	       OverflowError	PendingDeprecationWarning	      ReferenceError	        RuntimeError	      RuntimeWarning	       StandardError	       StopIteration	         SyntaxError	       SyntaxWarning	         SystemError	          SystemExit	            TabError	                True	           TypeError	   UnboundLocalError	  UnicodeDecodeError	  UnicodeEncodeError	        UnicodeError	UnicodeTranslateError	      UnicodeWarning	         UserWarning	          ValueError	             Warning	   ZeroDivisionError	                   _	           __debug__	             __doc__	          __import__	            __name__	         __package__	                 abs	                 all	                 any	               apply	          basestring	                 bin	                bool	              buffer	           bytearray	               bytes	            callable	                 chr	         classmethod	                 cmp	              coerce	             compile	             complex	           copyright	             credits	             delattr	                dict	                 dir	              divmod	           enumerate	                eval	            execfile	                exit	                file	              filter	               float	              format	           frozenset	             getattr	             globals	             hasattr	                hash	                help	                 hex	                  id	               input	                 int	              intern	          isinstance	          issubclass	                iter	                 len	             license	                list	              locals	                long	                 map	                 max	          memoryview	                 min	                next	              object	                 oct	                open	                 ord	                 pow	               print	            property	                quit	               range	           raw_input	              reduce	              reload	                repr	            reversed	               round	                 set	             setattr	               slice	              sorted	        staticmethod	                 str	                 sum	               super	               tuple	                type	              unichr	             unicode	                vars	              xrange	                 zip	>>>

　　

　　

 3. dir内置命令是怎么实现的

在/Python-2.7.8/Objects/object.c中

1963 /* Implementation of dir() -- if obj is NULL, returns the names in the current1964    (local) scope.  Otherwise, performs introspection of the object: returns a1965    sorted list of attribute names (supposedly) accessible from the object1966 */1967 PyObject *1968 PyObject_Dir(PyObject *obj)1969 {1970     PyObject * result;1971 1972     if (obj == NULL)1973         /* no object -- introspect the locals */1974         result = _dir_locals();1975     else1976         /* object -- introspect the object */1977         result = _dir_object(obj);1978 1979     assert(result == NULL || PyList_Check(result));1980 1981     if (result != NULL && PyList_Sort(result) != 0) {1982         /* sorting the list failed */1983         Py_DECREF(result);1984         result = NULL;1985     }1986 1987     return result;1988 }

　　

可见，与help(dir)描述的基本一致。

 

>>> def foo(a):...     if a > x:...             return a/1024...     else:...             return a... >>> type(foo)<type ‘function‘>>>> dir(foo)[‘__call__‘, ‘__class__‘, ‘__closure__‘, ‘__code__‘, ‘__defaults__‘, ‘__delattr__‘, ‘__dict__‘, ‘__doc__‘, ‘__format__‘, ‘__get__‘, ‘__getattribute__‘, ‘__globals__‘, ‘__hash__‘, ‘__init__‘, ‘__module__‘, ‘__name__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘func_closure‘, ‘func_code‘, ‘func_defaults‘, ‘func_dict‘, ‘func_doc‘, ‘func_globals‘, ‘func_name‘]>>> foo.__call__<method-wrapper ‘__call__‘ of function object at 0xb7420df4>>>> foo.__str__<method-wrapper ‘__str__‘ of function object at 0xb7420df4>>>> foo<function foo at 0xb7420df4>>>> foo.func_closure>>> type(foo.func_closure)<type ‘NoneType‘>>>> type(foo.func_code)<type ‘code‘>>>> foo.func_code<code object foo at 0xb7409d10, file "<stdin>", line 1>>>> dir(foo.func_code)[‘__class__‘, ‘__cmp__‘, ‘__delattr__‘, ‘__doc__‘, ‘__eq__‘, ‘__format__‘, ‘__ge__‘, ‘__getattribute__‘, ‘__gt__‘, ‘__hash__‘, ‘__init__‘, ‘__le__‘, ‘__lt__‘, ‘__ne__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘co_argcount‘, ‘co_cellvars‘, ‘co_code‘, ‘co_consts‘, ‘co_filename‘, ‘co_firstlineno‘, ‘co_flags‘, ‘co_freevars‘, ‘co_lnotab‘, ‘co_name‘, ‘co_names‘, ‘co_nlocals‘, ‘co_stacksize‘, ‘co_varnames‘]>>> foo.func_code.co_argcount1>>> foo.func_code.co_cellvars()>>> foo.func_code.co_code‘|\x00\x00t\x00\x00k\x04\x00r\x14\x00|\x00\x00d\x01\x00\x15S|\x00\x00Sd\x00\x00S‘>>> foo.func_code.co_consts(None, 1024)>>> foo.func_code.co_filename‘<stdin>‘>>> foo.func_code.co_firstlineno1>>> foo.func_code.co_flags67>>> foo.func_code.co_freevars()>>> foo.func_code.co_lnotab‘\x00\x01\x0c\x01\x08\x02‘>>> foo.func_code.co_name‘foo‘>>> foo.func_code.co_names(‘x‘,)>>> foo.func_code.co_nlocals1>>> foo.func_code.co_stacksize2>>> foo.func_code.co_varnames(‘a‘,)>>>

　　

其中，foo.func_code.co_code打印出来的就是Python的字节码。

 

Help on built-in function ord in module __builtin__:ord(...)    ord(c) -> integer        Return the integer ordinal of a one-character string.

　　

>>> [ord(i) for i in foo.func_code.co_code][124, 0, 0, 116, 0, 0, 107, 4, 0, 114, 20, 0, 124, 0, 0, 100, 1, 0, 21, 83, 124, 0, 0, 83, 100, 0, 0, 83]

 

这就是那些组成python字节码的字节。解释器会循环接收各个字节，查找每个字节的指令然后执行这个指令。需要注意的是，字节码本身并不包括任何python对象,或引用任何对象。

如果你想知道python字节码的意思，可以去找到CPython解释器文件(ceval.c)，然后查阅100的意思、1的意思、0的意思，等等。　　

 

>>> import dis>>> dir(dis)[‘EXTENDED_ARG‘, ‘HAVE_ARGUMENT‘, ‘__all__‘, ‘__builtins__‘, ‘__doc__‘, ‘__file__‘, ‘__name__‘, ‘__package__‘, ‘_have_code‘, ‘_test‘, ‘cmp_op‘, ‘dis‘, ‘disassemble‘, ‘disassemble_string‘, ‘disco‘, ‘distb‘, ‘findlabels‘, ‘findlinestarts‘, ‘hascompare‘, ‘hasconst‘, ‘hasfree‘, ‘hasjabs‘, ‘hasjrel‘, ‘haslocal‘, ‘hasname‘, ‘opmap‘, ‘opname‘, ‘sys‘, ‘types‘]>>> type(dis.dis)<type ‘function‘>>>> dir(dis.dis)[‘__call__‘, ‘__class__‘, ‘__closure__‘, ‘__code__‘, ‘__defaults__‘, ‘__delattr__‘, ‘__dict__‘, ‘__doc__‘, ‘__format__‘, ‘__get__‘, ‘__getattribute__‘, ‘__globals__‘, ‘__hash__‘, ‘__init__‘, ‘__module__‘, ‘__name__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘func_closure‘, ‘func_code‘, ‘func_defaults‘, ‘func_dict‘, ‘func_doc‘, ‘func_globals‘, ‘func_name‘]>>> [ord(i) for i in dis.dis.func_code.co_code][124, 0, 0, 100, 1, 0, 107, 8, 0, 114, 23, 0, 116, 1, 0, 131, 0, 0, 1, 100, 1, 0, 83, 116, 2, 0, 124, 0, 0, 116, 3, 0, 106, 4, 0, 131, 2, 0, 114, 53, 0, 124, 0, 0, 106, 5, 0, 125, 0, 0, 110, 0, 0, 116, 6, 0, 124, 0, 0, 100, 2, 0, 131, 2, 0, 114, 80, 0, 124, 0, 0, 106, 7, 0, 125, 0, 0, 110, 0, 0, 116, 6, 0, 124, 0, 0, 100, 3, 0, 131, 2, 0, 114, 107, 0, 124, 0, 0, 106, 8, 0, 125, 0, 0, 110, 0, 0, 116, 6, 0, 124, 0, 0, 100, 4, 0, 131, 2, 0, 114, 246, 0, 124, 0, 0, 106, 9, 0, 106, 10, 0, 131, 0, 0, 125, 1, 0, 124, 1, 0, 106, 11, 0, 131, 0, 0, 1, 120, 174, 0, 124, 1, 0, 68, 93, 85, 0, 92, 2, 0, 125, 2, 0, 125, 3, 0, 116, 2, 0, 124, 3, 0, 116, 12, 0, 131, 2, 0, 114, 154, 0, 100, 5, 0, 124, 2, 0, 22, 71, 72, 121, 14, 0, 116, 13, 0, 124, 3, 0, 131, 1, 0, 1, 87, 110, 28, 0, 4, 116, 14, 0, 107, 10, 0, 114, 234, 0, 1, 125, 4, 0, 1, 100, 6, 0, 71, 124, 4, 0, 71, 72, 110, 1, 0, 88, 72, 113, 154, 0, 113, 154, 0, 87, 110, 78, 0, 116, 6, 0, 124, 0, 0, 100, 7, 0, 131, 2, 0, 114, 18, 1, 116, 15, 0, 124, 0, 0, 131, 1, 0, 1, 110, 50, 0, 116, 2, 0, 124, 0, 0, 116, 16, 0, 131, 2, 0, 114, 46, 1, 116, 17, 0, 124, 0, 0, 131, 1, 0, 1, 110, 22, 0, 116, 14, 0, 100, 8, 0, 116, 18, 0, 124, 0, 0, 131, 1, 0, 106, 19, 0, 22, 130, 2, 0, 100, 1, 0, 83]

　　

>>> dir(dis.dis.func_code)[‘__class__‘, ‘__cmp__‘, ‘__delattr__‘, ‘__doc__‘, ‘__eq__‘, ‘__format__‘, ‘__ge__‘, ‘__getattribute__‘, ‘__gt__‘, ‘__hash__‘, ‘__init__‘, ‘__le__‘, ‘__lt__‘, ‘__ne__‘, ‘__new__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘, ‘__subclasshook__‘, ‘co_argcount‘, ‘co_cellvars‘, ‘co_code‘, ‘co_consts‘, ‘co_filename‘, ‘co_firstlineno‘, ‘co_flags‘, ‘co_freevars‘, ‘co_lnotab‘, ‘co_name‘, ‘co_names‘, ‘co_nlocals‘, ‘co_stacksize‘, ‘co_varnames‘]>>> dis.dis.func_code.co_filename‘/usr/lib/python2.7/dis.py‘>>> dis.dis.func_code.co_consts(‘Disassemble classes, methods, functions, or code.\n\n    With no argument, disassemble the last traceback.\n\n    ‘, None, ‘im_func‘, ‘func_code‘, ‘__dict__‘, ‘Disassembly of %s:‘, ‘Sorry:‘, ‘co_code‘, "don‘t know how to disassemble %s objects")>>> dis.dis.func_code.co_names(‘None‘, ‘distb‘, ‘isinstance‘, ‘types‘, ‘InstanceType‘, ‘__class__‘, ‘hasattr‘, ‘im_func‘, ‘func_code‘, ‘__dict__‘, ‘items‘, ‘sort‘, ‘_have_code‘, ‘dis‘, ‘TypeError‘, ‘disassemble‘, ‘str‘, ‘disassemble_string‘, ‘type‘, ‘__name__‘)>>> dis.dis.func_code.co_varnames(‘x‘, ‘items‘, ‘name‘, ‘x1‘, ‘msg‘)>>> dis.dis.func_code.co_stacksize6>>> dis.dis.func_code.co_nlocals5

　　

其实dis.dis也不过就是是一连串的字节码而已，它被Python解释器执行，从而完成指定的功能。

下面我们就使用dis.dis来反汇编一下字节码

>>> dis.dis(foo.func_code.co_code)          0 LOAD_FAST           0 (0)          3 LOAD_GLOBAL         0 (0)          6 COMPARE_OP          4 (>)          9 POP_JUMP_IF_FALSE    20         12 LOAD_FAST           0 (0)         15 LOAD_CONST          1 (1)         18 BINARY_DIVIDE           19 RETURN_VALUE       >>   20 LOAD_FAST           0 (0)         23 RETURN_VALUE            24 LOAD_CONST          0 (0)         27 RETURN_VALUE

　　

 

 

 

 

 

 

　　

 

 


联系
我们