AppPkg/Applications/Python/Python-2.7.10: Initial Checkin part 3/5.

The Objects directory from the cPython 2.7.10 distribution, along with the LICENSE and README files.  These files are unchanged and set the baseline for subsequent commits.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Daryl McDaniel <edk2-lists@mc2research.org>


git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@18739 6f19259b-4bc3-4df7-8a09-765794883524
This commit is contained in:
Daryl McDaniel
2015-11-07 19:29:24 +00:00
committed by darylm503
parent 7eb75bccb5
commit 53b2ba5790
56 changed files with 75760 additions and 0 deletions

View File

@ -0,0 +1,254 @@
A. HISTORY OF THE SOFTWARE
==========================
Python was created in the early 1990s by Guido van Rossum at Stichting
Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
as a successor of a language called ABC. Guido remains Python's
principal author, although it includes many contributions from others.
In 1995, Guido continued his work on Python at the Corporation for
National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
in Reston, Virginia where he released several versions of the
software.
In May 2000, Guido and the Python core development team moved to
BeOpen.com to form the BeOpen PythonLabs team. In October of the same
year, the PythonLabs team moved to Digital Creations (now Zope
Corporation, see http://www.zope.com). In 2001, the Python Software
Foundation (PSF, see http://www.python.org/psf/) was formed, a
non-profit organization created specifically to own Python-related
Intellectual Property. Zope Corporation is a sponsoring member of
the PSF.
All Python releases are Open Source (see http://www.opensource.org for
the Open Source Definition). Historically, most, but not all, Python
releases have also been GPL-compatible; the table below summarizes
the various releases.
Release Derived Year Owner GPL-
from compatible? (1)
0.9.0 thru 1.2 1991-1995 CWI yes
1.3 thru 1.5.2 1.2 1995-1999 CNRI yes
1.6 1.5.2 2000 CNRI no
2.0 1.6 2000 BeOpen.com no
1.6.1 1.6 2001 CNRI yes (2)
2.1 2.0+1.6.1 2001 PSF no
2.0.1 2.0+1.6.1 2001 PSF yes
2.1.1 2.1+2.0.1 2001 PSF yes
2.1.2 2.1.1 2002 PSF yes
2.1.3 2.1.2 2002 PSF yes
2.2 and above 2.1.1 2001-now PSF yes
Footnotes:
(1) GPL-compatible doesn't mean that we're distributing Python under
the GPL. All Python licenses, unlike the GPL, let you distribute
a modified version without making your changes open source. The
GPL-compatible licenses make it possible to combine Python with
other software that is released under the GPL; the others don't.
(2) According to Richard Stallman, 1.6.1 is not GPL-compatible,
because its license has a choice of law clause. According to
CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1
is "not incompatible" with the GPL.
Thanks to the many outside volunteers who have worked under Guido's
direction to make these releases possible.
B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
===============================================================
PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
--------------------------------------------
1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using this software ("Python") in source or binary form and
its associated documentation.
2. Subject to the terms and conditions of this License Agreement, PSF hereby
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
analyze, test, perform and/or display publicly, prepare derivative works,
distribute, and otherwise use Python alone or in any derivative version,
provided, however, that PSF's License Agreement and PSF's notice of copyright,
i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved"
are retained in Python alone or in any derivative version prepared by Licensee.
3. In the event Licensee prepares a derivative work that is based on
or incorporates Python or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python.
4. PSF is making Python available to Licensee on an "AS IS"
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee. This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.
8. By copying, installing or otherwise using Python, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.
BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
-------------------------------------------
BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
Individual or Organization ("Licensee") accessing and otherwise using
this software in source or binary form and its associated
documentation ("the Software").
2. Subject to the terms and conditions of this BeOpen Python License
Agreement, BeOpen hereby grants Licensee a non-exclusive,
royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and
otherwise use the Software alone or in any derivative version,
provided, however, that the BeOpen Python License is retained in the
Software, alone or in any derivative version prepared by Licensee.
3. BeOpen is making the Software available to Licensee on an "AS IS"
basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
5. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
6. This License Agreement shall be governed by and interpreted in all
respects by the law of the State of California, excluding conflict of
law provisions. Nothing in this License Agreement shall be deemed to
create any relationship of agency, partnership, or joint venture
between BeOpen and Licensee. This License Agreement does not grant
permission to use BeOpen trademarks or trade names in a trademark
sense to endorse or promote products or services of Licensee, or any
third party. As an exception, the "BeOpen Python" logos available at
http://www.pythonlabs.com/logos.html may be used according to the
permissions granted on that web page.
7. By copying, installing or otherwise using the software, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.
CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
---------------------------------------
1. This LICENSE AGREEMENT is between the Corporation for National
Research Initiatives, having an office at 1895 Preston White Drive,
Reston, VA 20191 ("CNRI"), and the Individual or Organization
("Licensee") accessing and otherwise using Python 1.6.1 software in
source or binary form and its associated documentation.
2. Subject to the terms and conditions of this License Agreement, CNRI
hereby grants Licensee a nonexclusive, royalty-free, world-wide
license to reproduce, analyze, test, perform and/or display publicly,
prepare derivative works, distribute, and otherwise use Python 1.6.1
alone or in any derivative version, provided, however, that CNRI's
License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
1995-2001 Corporation for National Research Initiatives; All Rights
Reserved" are retained in Python 1.6.1 alone or in any derivative
version prepared by Licensee. Alternately, in lieu of CNRI's License
Agreement, Licensee may substitute the following text (omitting the
quotes): "Python 1.6.1 is made available subject to the terms and
conditions in CNRI's License Agreement. This Agreement together with
Python 1.6.1 may be located on the Internet using the following
unique, persistent identifier (known as a handle): 1895.22/1013. This
Agreement may also be obtained from a proxy server on the Internet
using the following URL: http://hdl.handle.net/1895.22/1013".
3. In the event Licensee prepares a derivative work that is based on
or incorporates Python 1.6.1 or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python 1.6.1.
4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
7. This License Agreement shall be governed by the federal
intellectual property law of the United States, including without
limitation the federal copyright law, and, to the extent such
U.S. federal law does not apply, by the law of the Commonwealth of
Virginia, excluding Virginia's conflict of law provisions.
Notwithstanding the foregoing, with regard to derivative works based
on Python 1.6.1 that incorporate non-separable material that was
previously distributed under the GNU General Public License (GPL), the
law of the Commonwealth of Virginia shall govern this License
Agreement only as to issues arising under or with respect to
Paragraphs 4, 5, and 7 of this License Agreement. Nothing in this
License Agreement shall be deemed to create any relationship of
agency, partnership, or joint venture between CNRI and Licensee. This
License Agreement does not grant permission to use CNRI trademarks or
trade name in a trademark sense to endorse or promote products or
services of Licensee, or any third party.
8. By clicking on the "ACCEPT" button where indicated, or by copying,
installing or otherwise using Python 1.6.1, Licensee agrees to be
bound by the terms and conditions of this License Agreement.
ACCEPT
CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
--------------------------------------------------
Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
The Netherlands. All rights reserved.
Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Stichting Mathematisch
Centrum or CWI not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.
STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,202 @@
/* Boolean type, a subtype of int */
#include "Python.h"
/* We need to define bool_print to override int_print */
static int
bool_print(PyBoolObject *self, FILE *fp, int flags)
{
Py_BEGIN_ALLOW_THREADS
fputs(self->ob_ival == 0 ? "False" : "True", fp);
Py_END_ALLOW_THREADS
return 0;
}
/* We define bool_repr to return "False" or "True" */
static PyObject *false_str = NULL;
static PyObject *true_str = NULL;
static PyObject *
bool_repr(PyBoolObject *self)
{
PyObject *s;
if (self->ob_ival)
s = true_str ? true_str :
(true_str = PyString_InternFromString("True"));
else
s = false_str ? false_str :
(false_str = PyString_InternFromString("False"));
Py_XINCREF(s);
return s;
}
/* Function to return a bool from a C long */
PyObject *PyBool_FromLong(long ok)
{
PyObject *result;
if (ok)
result = Py_True;
else
result = Py_False;
Py_INCREF(result);
return result;
}
/* We define bool_new to always return either Py_True or Py_False */
static PyObject *
bool_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {"x", 0};
PyObject *x = Py_False;
long ok;
if (!PyArg_ParseTupleAndKeywords(args, kwds, "|O:bool", kwlist, &x))
return NULL;
ok = PyObject_IsTrue(x);
if (ok < 0)
return NULL;
return PyBool_FromLong(ok);
}
/* Arithmetic operations redefined to return bool if both args are bool. */
static PyObject *
bool_and(PyObject *a, PyObject *b)
{
if (!PyBool_Check(a) || !PyBool_Check(b))
return PyInt_Type.tp_as_number->nb_and(a, b);
return PyBool_FromLong(
((PyBoolObject *)a)->ob_ival & ((PyBoolObject *)b)->ob_ival);
}
static PyObject *
bool_or(PyObject *a, PyObject *b)
{
if (!PyBool_Check(a) || !PyBool_Check(b))
return PyInt_Type.tp_as_number->nb_or(a, b);
return PyBool_FromLong(
((PyBoolObject *)a)->ob_ival | ((PyBoolObject *)b)->ob_ival);
}
static PyObject *
bool_xor(PyObject *a, PyObject *b)
{
if (!PyBool_Check(a) || !PyBool_Check(b))
return PyInt_Type.tp_as_number->nb_xor(a, b);
return PyBool_FromLong(
((PyBoolObject *)a)->ob_ival ^ ((PyBoolObject *)b)->ob_ival);
}
/* Doc string */
PyDoc_STRVAR(bool_doc,
"bool(x) -> bool\n\
\n\
Returns True when the argument x is true, False otherwise.\n\
The builtins True and False are the only two instances of the class bool.\n\
The class bool is a subclass of the class int, and cannot be subclassed.");
/* Arithmetic methods -- only so we can override &, |, ^. */
static PyNumberMethods bool_as_number = {
0, /* nb_add */
0, /* nb_subtract */
0, /* nb_multiply */
0, /* nb_divide */
0, /* nb_remainder */
0, /* nb_divmod */
0, /* nb_power */
0, /* nb_negative */
0, /* nb_positive */
0, /* nb_absolute */
0, /* nb_nonzero */
0, /* nb_invert */
0, /* nb_lshift */
0, /* nb_rshift */
bool_and, /* nb_and */
bool_xor, /* nb_xor */
bool_or, /* nb_or */
0, /* nb_coerce */
0, /* nb_int */
0, /* nb_long */
0, /* nb_float */
0, /* nb_oct */
0, /* nb_hex */
0, /* nb_inplace_add */
0, /* nb_inplace_subtract */
0, /* nb_inplace_multiply */
0, /* nb_inplace_divide */
0, /* nb_inplace_remainder */
0, /* nb_inplace_power */
0, /* nb_inplace_lshift */
0, /* nb_inplace_rshift */
0, /* nb_inplace_and */
0, /* nb_inplace_xor */
0, /* nb_inplace_or */
0, /* nb_floor_divide */
0, /* nb_true_divide */
0, /* nb_inplace_floor_divide */
0, /* nb_inplace_true_divide */
};
/* The type object for bool. Note that this cannot be subclassed! */
PyTypeObject PyBool_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"bool",
sizeof(PyIntObject),
0,
0, /* tp_dealloc */
(printfunc)bool_print, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)bool_repr, /* tp_repr */
&bool_as_number, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
(reprfunc)bool_repr, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES, /* tp_flags */
bool_doc, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
&PyInt_Type, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
bool_new, /* tp_new */
};
/* The objects representing bool values False and True */
/* Named Zero for link-level compatibility */
PyIntObject _Py_ZeroStruct = {
PyObject_HEAD_INIT(&PyBool_Type)
0
};
PyIntObject _Py_TrueStruct = {
PyObject_HEAD_INIT(&PyBool_Type)
1
};

View File

@ -0,0 +1,878 @@
/* Buffer object implementation */
#include "Python.h"
typedef struct {
PyObject_HEAD
PyObject *b_base;
void *b_ptr;
Py_ssize_t b_size;
Py_ssize_t b_offset;
int b_readonly;
long b_hash;
} PyBufferObject;
enum buffer_t {
READ_BUFFER,
WRITE_BUFFER,
CHAR_BUFFER,
ANY_BUFFER
};
static int
get_buf(PyBufferObject *self, void **ptr, Py_ssize_t *size,
enum buffer_t buffer_type)
{
if (self->b_base == NULL) {
assert (ptr != NULL);
*ptr = self->b_ptr;
*size = self->b_size;
}
else {
Py_ssize_t count, offset;
readbufferproc proc = 0;
PyBufferProcs *bp = self->b_base->ob_type->tp_as_buffer;
if ((*bp->bf_getsegcount)(self->b_base, NULL) != 1) {
PyErr_SetString(PyExc_TypeError,
"single-segment buffer object expected");
return 0;
}
if ((buffer_type == READ_BUFFER) ||
((buffer_type == ANY_BUFFER) && self->b_readonly))
proc = bp->bf_getreadbuffer;
else if ((buffer_type == WRITE_BUFFER) ||
(buffer_type == ANY_BUFFER))
proc = (readbufferproc)bp->bf_getwritebuffer;
else if (buffer_type == CHAR_BUFFER) {
if (!PyType_HasFeature(self->ob_type,
Py_TPFLAGS_HAVE_GETCHARBUFFER)) {
PyErr_SetString(PyExc_TypeError,
"Py_TPFLAGS_HAVE_GETCHARBUFFER needed");
return 0;
}
proc = (readbufferproc)bp->bf_getcharbuffer;
}
if (!proc) {
char *buffer_type_name;
switch (buffer_type) {
case READ_BUFFER:
buffer_type_name = "read";
break;
case WRITE_BUFFER:
buffer_type_name = "write";
break;
case CHAR_BUFFER:
buffer_type_name = "char";
break;
default:
buffer_type_name = "no";
break;
}
PyErr_Format(PyExc_TypeError,
"%s buffer type not available",
buffer_type_name);
return 0;
}
if ((count = (*proc)(self->b_base, 0, ptr)) < 0)
return 0;
/* apply constraints to the start/end */
if (self->b_offset > count)
offset = count;
else
offset = self->b_offset;
*(char **)ptr = *(char **)ptr + offset;
if (self->b_size == Py_END_OF_BUFFER)
*size = count;
else
*size = self->b_size;
if (*size > count - offset)
*size = count - offset;
}
return 1;
}
static PyObject *
buffer_from_memory(PyObject *base, Py_ssize_t size, Py_ssize_t offset, void *ptr,
int readonly)
{
PyBufferObject * b;
if (size < 0 && size != Py_END_OF_BUFFER) {
PyErr_SetString(PyExc_ValueError,
"size must be zero or positive");
return NULL;
}
if (offset < 0) {
PyErr_SetString(PyExc_ValueError,
"offset must be zero or positive");
return NULL;
}
b = PyObject_NEW(PyBufferObject, &PyBuffer_Type);
if ( b == NULL )
return NULL;
Py_XINCREF(base);
b->b_base = base;
b->b_ptr = ptr;
b->b_size = size;
b->b_offset = offset;
b->b_readonly = readonly;
b->b_hash = -1;
return (PyObject *) b;
}
static PyObject *
buffer_from_object(PyObject *base, Py_ssize_t size, Py_ssize_t offset, int readonly)
{
if (offset < 0) {
PyErr_SetString(PyExc_ValueError,
"offset must be zero or positive");
return NULL;
}
if ( PyBuffer_Check(base) && (((PyBufferObject *)base)->b_base) ) {
/* another buffer, refer to the base object */
PyBufferObject *b = (PyBufferObject *)base;
if (b->b_size != Py_END_OF_BUFFER) {
Py_ssize_t base_size = b->b_size - offset;
if (base_size < 0)
base_size = 0;
if (size == Py_END_OF_BUFFER || size > base_size)
size = base_size;
}
offset += b->b_offset;
base = b->b_base;
}
return buffer_from_memory(base, size, offset, NULL, readonly);
}
PyObject *
PyBuffer_FromObject(PyObject *base, Py_ssize_t offset, Py_ssize_t size)
{
PyBufferProcs *pb = base->ob_type->tp_as_buffer;
if ( pb == NULL ||
pb->bf_getreadbuffer == NULL ||
pb->bf_getsegcount == NULL )
{
PyErr_SetString(PyExc_TypeError, "buffer object expected");
return NULL;
}
return buffer_from_object(base, size, offset, 1);
}
PyObject *
PyBuffer_FromReadWriteObject(PyObject *base, Py_ssize_t offset, Py_ssize_t size)
{
PyBufferProcs *pb = base->ob_type->tp_as_buffer;
if ( pb == NULL ||
pb->bf_getwritebuffer == NULL ||
pb->bf_getsegcount == NULL )
{
PyErr_SetString(PyExc_TypeError, "buffer object expected");
return NULL;
}
return buffer_from_object(base, size, offset, 0);
}
PyObject *
PyBuffer_FromMemory(void *ptr, Py_ssize_t size)
{
return buffer_from_memory(NULL, size, 0, ptr, 1);
}
PyObject *
PyBuffer_FromReadWriteMemory(void *ptr, Py_ssize_t size)
{
return buffer_from_memory(NULL, size, 0, ptr, 0);
}
PyObject *
PyBuffer_New(Py_ssize_t size)
{
PyObject *o;
PyBufferObject * b;
if (size < 0) {
PyErr_SetString(PyExc_ValueError,
"size must be zero or positive");
return NULL;
}
if (sizeof(*b) > PY_SSIZE_T_MAX - size) {
/* unlikely */
return PyErr_NoMemory();
}
/* Inline PyObject_New */
o = (PyObject *)PyObject_MALLOC(sizeof(*b) + size);
if ( o == NULL )
return PyErr_NoMemory();
b = (PyBufferObject *) PyObject_INIT(o, &PyBuffer_Type);
b->b_base = NULL;
b->b_ptr = (void *)(b + 1);
b->b_size = size;
b->b_offset = 0;
b->b_readonly = 0;
b->b_hash = -1;
return o;
}
/* Methods */
static PyObject *
buffer_new(PyTypeObject *type, PyObject *args, PyObject *kw)
{
PyObject *ob;
Py_ssize_t offset = 0;
Py_ssize_t size = Py_END_OF_BUFFER;
if (PyErr_WarnPy3k("buffer() not supported in 3.x", 1) < 0)
return NULL;
if (!_PyArg_NoKeywords("buffer()", kw))
return NULL;
if (!PyArg_ParseTuple(args, "O|nn:buffer", &ob, &offset, &size))
return NULL;
return PyBuffer_FromObject(ob, offset, size);
}
PyDoc_STRVAR(buffer_doc,
"buffer(object [, offset[, size]])\n\
\n\
Create a new buffer object which references the given object.\n\
The buffer will reference a slice of the target object from the\n\
start of the object (or at the specified offset). The slice will\n\
extend to the end of the target object (or with the specified size).");
static void
buffer_dealloc(PyBufferObject *self)
{
Py_XDECREF(self->b_base);
PyObject_DEL(self);
}
static int
buffer_compare(PyBufferObject *self, PyBufferObject *other)
{
void *p1, *p2;
Py_ssize_t len_self, len_other, min_len;
int cmp;
if (!get_buf(self, &p1, &len_self, ANY_BUFFER))
return -1;
if (!get_buf(other, &p2, &len_other, ANY_BUFFER))
return -1;
min_len = (len_self < len_other) ? len_self : len_other;
if (min_len > 0) {
cmp = memcmp(p1, p2, min_len);
if (cmp != 0)
return cmp < 0 ? -1 : 1;
}
return (len_self < len_other) ? -1 : (len_self > len_other) ? 1 : 0;
}
static PyObject *
buffer_repr(PyBufferObject *self)
{
const char *status = self->b_readonly ? "read-only" : "read-write";
if ( self->b_base == NULL )
return PyString_FromFormat("<%s buffer ptr %p, size %zd at %p>",
status,
self->b_ptr,
self->b_size,
self);
else
return PyString_FromFormat(
"<%s buffer for %p, size %zd, offset %zd at %p>",
status,
self->b_base,
self->b_size,
self->b_offset,
self);
}
static long
buffer_hash(PyBufferObject *self)
{
void *ptr;
Py_ssize_t size;
register Py_ssize_t len;
register unsigned char *p;
register long x;
if ( self->b_hash != -1 )
return self->b_hash;
/* XXX potential bugs here, a readonly buffer does not imply that the
* underlying memory is immutable. b_readonly is a necessary but not
* sufficient condition for a buffer to be hashable. Perhaps it would
* be better to only allow hashing if the underlying object is known to
* be immutable (e.g. PyString_Check() is true). Another idea would
* be to call tp_hash on the underlying object and see if it raises
* an error. */
if ( !self->b_readonly )
{
PyErr_SetString(PyExc_TypeError,
"writable buffers are not hashable");
return -1;
}
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return -1;
p = (unsigned char *) ptr;
len = size;
/*
We make the hash of the empty buffer be 0, rather than using
(prefix ^ suffix), since this slightly obfuscates the hash secret
*/
if (len == 0) {
self->b_hash = 0;
return 0;
}
x = _Py_HashSecret.prefix;
x ^= *p << 7;
while (--len >= 0)
x = (1000003*x) ^ *p++;
x ^= size;
x ^= _Py_HashSecret.suffix;
if (x == -1)
x = -2;
self->b_hash = x;
return x;
}
static PyObject *
buffer_str(PyBufferObject *self)
{
void *ptr;
Py_ssize_t size;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return NULL;
return PyString_FromStringAndSize((const char *)ptr, size);
}
/* Sequence methods */
static Py_ssize_t
buffer_length(PyBufferObject *self)
{
void *ptr;
Py_ssize_t size;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return -1;
return size;
}
static PyObject *
buffer_concat(PyBufferObject *self, PyObject *other)
{
PyBufferProcs *pb = other->ob_type->tp_as_buffer;
void *ptr1, *ptr2;
char *p;
PyObject *ob;
Py_ssize_t size, count;
if ( pb == NULL ||
pb->bf_getreadbuffer == NULL ||
pb->bf_getsegcount == NULL )
{
PyErr_BadArgument();
return NULL;
}
if ( (*pb->bf_getsegcount)(other, NULL) != 1 )
{
/* ### use a different exception type/message? */
PyErr_SetString(PyExc_TypeError,
"single-segment buffer object expected");
return NULL;
}
if (!get_buf(self, &ptr1, &size, ANY_BUFFER))
return NULL;
/* optimize special case */
if ( size == 0 )
{
Py_INCREF(other);
return other;
}
if ( (count = (*pb->bf_getreadbuffer)(other, 0, &ptr2)) < 0 )
return NULL;
assert(count <= PY_SIZE_MAX - size);
ob = PyString_FromStringAndSize(NULL, size + count);
if ( ob == NULL )
return NULL;
p = PyString_AS_STRING(ob);
memcpy(p, ptr1, size);
memcpy(p + size, ptr2, count);
/* there is an extra byte in the string object, so this is safe */
p[size + count] = '\0';
return ob;
}
static PyObject *
buffer_repeat(PyBufferObject *self, Py_ssize_t count)
{
PyObject *ob;
register char *p;
void *ptr;
Py_ssize_t size;
if ( count < 0 )
count = 0;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return NULL;
if (count > PY_SSIZE_T_MAX / size) {
PyErr_SetString(PyExc_MemoryError, "result too large");
return NULL;
}
ob = PyString_FromStringAndSize(NULL, size * count);
if ( ob == NULL )
return NULL;
p = PyString_AS_STRING(ob);
while ( count-- )
{
memcpy(p, ptr, size);
p += size;
}
/* there is an extra byte in the string object, so this is safe */
*p = '\0';
return ob;
}
static PyObject *
buffer_item(PyBufferObject *self, Py_ssize_t idx)
{
void *ptr;
Py_ssize_t size;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return NULL;
if ( idx < 0 || idx >= size ) {
PyErr_SetString(PyExc_IndexError, "buffer index out of range");
return NULL;
}
return PyString_FromStringAndSize((char *)ptr + idx, 1);
}
static PyObject *
buffer_slice(PyBufferObject *self, Py_ssize_t left, Py_ssize_t right)
{
void *ptr;
Py_ssize_t size;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return NULL;
if ( left < 0 )
left = 0;
if ( right < 0 )
right = 0;
if ( right > size )
right = size;
if ( right < left )
right = left;
return PyString_FromStringAndSize((char *)ptr + left,
right - left);
}
static PyObject *
buffer_subscript(PyBufferObject *self, PyObject *item)
{
void *p;
Py_ssize_t size;
if (!get_buf(self, &p, &size, ANY_BUFFER))
return NULL;
if (PyIndex_Check(item)) {
Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
if (i == -1 && PyErr_Occurred())
return NULL;
if (i < 0)
i += size;
return buffer_item(self, i);
}
else if (PySlice_Check(item)) {
Py_ssize_t start, stop, step, slicelength, cur, i;
if (PySlice_GetIndicesEx((PySliceObject*)item, size,
&start, &stop, &step, &slicelength) < 0) {
return NULL;
}
if (slicelength <= 0)
return PyString_FromStringAndSize("", 0);
else if (step == 1)
return PyString_FromStringAndSize((char *)p + start,
stop - start);
else {
PyObject *result;
char *source_buf = (char *)p;
char *result_buf = (char *)PyMem_Malloc(slicelength);
if (result_buf == NULL)
return PyErr_NoMemory();
for (cur = start, i = 0; i < slicelength;
cur += step, i++) {
result_buf[i] = source_buf[cur];
}
result = PyString_FromStringAndSize(result_buf,
slicelength);
PyMem_Free(result_buf);
return result;
}
}
else {
PyErr_SetString(PyExc_TypeError,
"sequence index must be integer");
return NULL;
}
}
static int
buffer_ass_item(PyBufferObject *self, Py_ssize_t idx, PyObject *other)
{
PyBufferProcs *pb;
void *ptr1, *ptr2;
Py_ssize_t size;
Py_ssize_t count;
if ( self->b_readonly ) {
PyErr_SetString(PyExc_TypeError,
"buffer is read-only");
return -1;
}
if (!get_buf(self, &ptr1, &size, ANY_BUFFER))
return -1;
if (idx < 0 || idx >= size) {
PyErr_SetString(PyExc_IndexError,
"buffer assignment index out of range");
return -1;
}
pb = other ? other->ob_type->tp_as_buffer : NULL;
if ( pb == NULL ||
pb->bf_getreadbuffer == NULL ||
pb->bf_getsegcount == NULL )
{
PyErr_BadArgument();
return -1;
}
if ( (*pb->bf_getsegcount)(other, NULL) != 1 )
{
/* ### use a different exception type/message? */
PyErr_SetString(PyExc_TypeError,
"single-segment buffer object expected");
return -1;
}
if ( (count = (*pb->bf_getreadbuffer)(other, 0, &ptr2)) < 0 )
return -1;
if ( count != 1 ) {
PyErr_SetString(PyExc_TypeError,
"right operand must be a single byte");
return -1;
}
((char *)ptr1)[idx] = *(char *)ptr2;
return 0;
}
static int
buffer_ass_slice(PyBufferObject *self, Py_ssize_t left, Py_ssize_t right, PyObject *other)
{
PyBufferProcs *pb;
void *ptr1, *ptr2;
Py_ssize_t size;
Py_ssize_t slice_len;
Py_ssize_t count;
if ( self->b_readonly ) {
PyErr_SetString(PyExc_TypeError,
"buffer is read-only");
return -1;
}
pb = other ? other->ob_type->tp_as_buffer : NULL;
if ( pb == NULL ||
pb->bf_getreadbuffer == NULL ||
pb->bf_getsegcount == NULL )
{
PyErr_BadArgument();
return -1;
}
if ( (*pb->bf_getsegcount)(other, NULL) != 1 )
{
/* ### use a different exception type/message? */
PyErr_SetString(PyExc_TypeError,
"single-segment buffer object expected");
return -1;
}
if (!get_buf(self, &ptr1, &size, ANY_BUFFER))
return -1;
if ( (count = (*pb->bf_getreadbuffer)(other, 0, &ptr2)) < 0 )
return -1;
if ( left < 0 )
left = 0;
else if ( left > size )
left = size;
if ( right < left )
right = left;
else if ( right > size )
right = size;
slice_len = right - left;
if ( count != slice_len ) {
PyErr_SetString(
PyExc_TypeError,
"right operand length must match slice length");
return -1;
}
if ( slice_len )
memcpy((char *)ptr1 + left, ptr2, slice_len);
return 0;
}
static int
buffer_ass_subscript(PyBufferObject *self, PyObject *item, PyObject *value)
{
PyBufferProcs *pb;
void *ptr1, *ptr2;
Py_ssize_t selfsize;
Py_ssize_t othersize;
if ( self->b_readonly ) {
PyErr_SetString(PyExc_TypeError,
"buffer is read-only");
return -1;
}
pb = value ? value->ob_type->tp_as_buffer : NULL;
if ( pb == NULL ||
pb->bf_getreadbuffer == NULL ||
pb->bf_getsegcount == NULL )
{
PyErr_BadArgument();
return -1;
}
if ( (*pb->bf_getsegcount)(value, NULL) != 1 )
{
/* ### use a different exception type/message? */
PyErr_SetString(PyExc_TypeError,
"single-segment buffer object expected");
return -1;
}
if (!get_buf(self, &ptr1, &selfsize, ANY_BUFFER))
return -1;
if (PyIndex_Check(item)) {
Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
if (i == -1 && PyErr_Occurred())
return -1;
if (i < 0)
i += selfsize;
return buffer_ass_item(self, i, value);
}
else if (PySlice_Check(item)) {
Py_ssize_t start, stop, step, slicelength;
if (PySlice_GetIndicesEx((PySliceObject *)item, selfsize,
&start, &stop, &step, &slicelength) < 0)
return -1;
if ((othersize = (*pb->bf_getreadbuffer)(value, 0, &ptr2)) < 0)
return -1;
if (othersize != slicelength) {
PyErr_SetString(
PyExc_TypeError,
"right operand length must match slice length");
return -1;
}
if (slicelength == 0)
return 0;
else if (step == 1) {
memcpy((char *)ptr1 + start, ptr2, slicelength);
return 0;
}
else {
Py_ssize_t cur, i;
for (cur = start, i = 0; i < slicelength;
cur += step, i++) {
((char *)ptr1)[cur] = ((char *)ptr2)[i];
}
return 0;
}
} else {
PyErr_SetString(PyExc_TypeError,
"buffer indices must be integers");
return -1;
}
}
/* Buffer methods */
static Py_ssize_t
buffer_getreadbuf(PyBufferObject *self, Py_ssize_t idx, void **pp)
{
Py_ssize_t size;
if ( idx != 0 ) {
PyErr_SetString(PyExc_SystemError,
"accessing non-existent buffer segment");
return -1;
}
if (!get_buf(self, pp, &size, READ_BUFFER))
return -1;
return size;
}
static Py_ssize_t
buffer_getwritebuf(PyBufferObject *self, Py_ssize_t idx, void **pp)
{
Py_ssize_t size;
if ( self->b_readonly )
{
PyErr_SetString(PyExc_TypeError, "buffer is read-only");
return -1;
}
if ( idx != 0 ) {
PyErr_SetString(PyExc_SystemError,
"accessing non-existent buffer segment");
return -1;
}
if (!get_buf(self, pp, &size, WRITE_BUFFER))
return -1;
return size;
}
static Py_ssize_t
buffer_getsegcount(PyBufferObject *self, Py_ssize_t *lenp)
{
void *ptr;
Py_ssize_t size;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return -1;
if (lenp)
*lenp = size;
return 1;
}
static Py_ssize_t
buffer_getcharbuf(PyBufferObject *self, Py_ssize_t idx, const char **pp)
{
void *ptr;
Py_ssize_t size;
if ( idx != 0 ) {
PyErr_SetString(PyExc_SystemError,
"accessing non-existent buffer segment");
return -1;
}
if (!get_buf(self, &ptr, &size, CHAR_BUFFER))
return -1;
*pp = (const char *)ptr;
return size;
}
static int buffer_getbuffer(PyBufferObject *self, Py_buffer *buf, int flags)
{
void *ptr;
Py_ssize_t size;
if (!get_buf(self, &ptr, &size, ANY_BUFFER))
return -1;
return PyBuffer_FillInfo(buf, (PyObject*)self, ptr, size,
self->b_readonly, flags);
}
static PySequenceMethods buffer_as_sequence = {
(lenfunc)buffer_length, /*sq_length*/
(binaryfunc)buffer_concat, /*sq_concat*/
(ssizeargfunc)buffer_repeat, /*sq_repeat*/
(ssizeargfunc)buffer_item, /*sq_item*/
(ssizessizeargfunc)buffer_slice, /*sq_slice*/
(ssizeobjargproc)buffer_ass_item, /*sq_ass_item*/
(ssizessizeobjargproc)buffer_ass_slice, /*sq_ass_slice*/
};
static PyMappingMethods buffer_as_mapping = {
(lenfunc)buffer_length,
(binaryfunc)buffer_subscript,
(objobjargproc)buffer_ass_subscript,
};
static PyBufferProcs buffer_as_buffer = {
(readbufferproc)buffer_getreadbuf,
(writebufferproc)buffer_getwritebuf,
(segcountproc)buffer_getsegcount,
(charbufferproc)buffer_getcharbuf,
(getbufferproc)buffer_getbuffer,
};
PyTypeObject PyBuffer_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"buffer",
sizeof(PyBufferObject),
0,
(destructor)buffer_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
(cmpfunc)buffer_compare, /* tp_compare */
(reprfunc)buffer_repr, /* tp_repr */
0, /* tp_as_number */
&buffer_as_sequence, /* tp_as_sequence */
&buffer_as_mapping, /* tp_as_mapping */
(hashfunc)buffer_hash, /* tp_hash */
0, /* tp_call */
(reprfunc)buffer_str, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
&buffer_as_buffer, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GETCHARBUFFER | Py_TPFLAGS_HAVE_NEWBUFFER, /* tp_flags */
buffer_doc, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
buffer_new, /* tp_new */
};

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,398 @@
#include "Python.h"
#include "bytes_methods.h"
PyDoc_STRVAR_shared(_Py_isspace__doc__,
"B.isspace() -> bool\n\
\n\
Return True if all characters in B are whitespace\n\
and there is at least one character in B, False otherwise.");
PyObject*
_Py_bytes_isspace(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
/* Shortcut for single character strings */
if (len == 1 && Py_ISSPACE(*p))
Py_RETURN_TRUE;
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
for (; p < e; p++) {
if (!Py_ISSPACE(*p))
Py_RETURN_FALSE;
}
Py_RETURN_TRUE;
}
PyDoc_STRVAR_shared(_Py_isalpha__doc__,
"B.isalpha() -> bool\n\
\n\
Return True if all characters in B are alphabetic\n\
and there is at least one character in B, False otherwise.");
PyObject*
_Py_bytes_isalpha(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
/* Shortcut for single character strings */
if (len == 1 && Py_ISALPHA(*p))
Py_RETURN_TRUE;
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
for (; p < e; p++) {
if (!Py_ISALPHA(*p))
Py_RETURN_FALSE;
}
Py_RETURN_TRUE;
}
PyDoc_STRVAR_shared(_Py_isalnum__doc__,
"B.isalnum() -> bool\n\
\n\
Return True if all characters in B are alphanumeric\n\
and there is at least one character in B, False otherwise.");
PyObject*
_Py_bytes_isalnum(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
/* Shortcut for single character strings */
if (len == 1 && Py_ISALNUM(*p))
Py_RETURN_TRUE;
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
for (; p < e; p++) {
if (!Py_ISALNUM(*p))
Py_RETURN_FALSE;
}
Py_RETURN_TRUE;
}
PyDoc_STRVAR_shared(_Py_isdigit__doc__,
"B.isdigit() -> bool\n\
\n\
Return True if all characters in B are digits\n\
and there is at least one character in B, False otherwise.");
PyObject*
_Py_bytes_isdigit(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
/* Shortcut for single character strings */
if (len == 1 && Py_ISDIGIT(*p))
Py_RETURN_TRUE;
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
for (; p < e; p++) {
if (!Py_ISDIGIT(*p))
Py_RETURN_FALSE;
}
Py_RETURN_TRUE;
}
PyDoc_STRVAR_shared(_Py_islower__doc__,
"B.islower() -> bool\n\
\n\
Return True if all cased characters in B are lowercase and there is\n\
at least one cased character in B, False otherwise.");
PyObject*
_Py_bytes_islower(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
int cased;
/* Shortcut for single character strings */
if (len == 1)
return PyBool_FromLong(Py_ISLOWER(*p));
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
cased = 0;
for (; p < e; p++) {
if (Py_ISUPPER(*p))
Py_RETURN_FALSE;
else if (!cased && Py_ISLOWER(*p))
cased = 1;
}
return PyBool_FromLong(cased);
}
PyDoc_STRVAR_shared(_Py_isupper__doc__,
"B.isupper() -> bool\n\
\n\
Return True if all cased characters in B are uppercase and there is\n\
at least one cased character in B, False otherwise.");
PyObject*
_Py_bytes_isupper(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
int cased;
/* Shortcut for single character strings */
if (len == 1)
return PyBool_FromLong(Py_ISUPPER(*p));
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
cased = 0;
for (; p < e; p++) {
if (Py_ISLOWER(*p))
Py_RETURN_FALSE;
else if (!cased && Py_ISUPPER(*p))
cased = 1;
}
return PyBool_FromLong(cased);
}
PyDoc_STRVAR_shared(_Py_istitle__doc__,
"B.istitle() -> bool\n\
\n\
Return True if B is a titlecased string and there is at least one\n\
character in B, i.e. uppercase characters may only follow uncased\n\
characters and lowercase characters only cased ones. Return False\n\
otherwise.");
PyObject*
_Py_bytes_istitle(const char *cptr, Py_ssize_t len)
{
register const unsigned char *p
= (unsigned char *) cptr;
register const unsigned char *e;
int cased, previous_is_cased;
/* Shortcut for single character strings */
if (len == 1)
return PyBool_FromLong(Py_ISUPPER(*p));
/* Special case for empty strings */
if (len == 0)
Py_RETURN_FALSE;
e = p + len;
cased = 0;
previous_is_cased = 0;
for (; p < e; p++) {
register const unsigned char ch = *p;
if (Py_ISUPPER(ch)) {
if (previous_is_cased)
Py_RETURN_FALSE;
previous_is_cased = 1;
cased = 1;
}
else if (Py_ISLOWER(ch)) {
if (!previous_is_cased)
Py_RETURN_FALSE;
previous_is_cased = 1;
cased = 1;
}
else
previous_is_cased = 0;
}
return PyBool_FromLong(cased);
}
PyDoc_STRVAR_shared(_Py_lower__doc__,
"B.lower() -> copy of B\n\
\n\
Return a copy of B with all ASCII characters converted to lowercase.");
void
_Py_bytes_lower(char *result, const char *cptr, Py_ssize_t len)
{
Py_ssize_t i;
/*
newobj = PyString_FromStringAndSize(NULL, len);
if (!newobj)
return NULL;
s = PyString_AS_STRING(newobj);
*/
Py_MEMCPY(result, cptr, len);
for (i = 0; i < len; i++) {
int c = Py_CHARMASK(result[i]);
if (Py_ISUPPER(c))
result[i] = Py_TOLOWER(c);
}
}
PyDoc_STRVAR_shared(_Py_upper__doc__,
"B.upper() -> copy of B\n\
\n\
Return a copy of B with all ASCII characters converted to uppercase.");
void
_Py_bytes_upper(char *result, const char *cptr, Py_ssize_t len)
{
Py_ssize_t i;
/*
newobj = PyString_FromStringAndSize(NULL, len);
if (!newobj)
return NULL;
s = PyString_AS_STRING(newobj);
*/
Py_MEMCPY(result, cptr, len);
for (i = 0; i < len; i++) {
int c = Py_CHARMASK(result[i]);
if (Py_ISLOWER(c))
result[i] = Py_TOUPPER(c);
}
}
PyDoc_STRVAR_shared(_Py_title__doc__,
"B.title() -> copy of B\n\
\n\
Return a titlecased version of B, i.e. ASCII words start with uppercase\n\
characters, all remaining cased characters have lowercase.");
void
_Py_bytes_title(char *result, char *s, Py_ssize_t len)
{
Py_ssize_t i;
int previous_is_cased = 0;
/*
newobj = PyString_FromStringAndSize(NULL, len);
if (newobj == NULL)
return NULL;
s_new = PyString_AsString(newobj);
*/
for (i = 0; i < len; i++) {
int c = Py_CHARMASK(*s++);
if (Py_ISLOWER(c)) {
if (!previous_is_cased)
c = Py_TOUPPER(c);
previous_is_cased = 1;
} else if (Py_ISUPPER(c)) {
if (previous_is_cased)
c = Py_TOLOWER(c);
previous_is_cased = 1;
} else
previous_is_cased = 0;
*result++ = c;
}
}
PyDoc_STRVAR_shared(_Py_capitalize__doc__,
"B.capitalize() -> copy of B\n\
\n\
Return a copy of B with only its first character capitalized (ASCII)\n\
and the rest lower-cased.");
void
_Py_bytes_capitalize(char *result, char *s, Py_ssize_t len)
{
Py_ssize_t i;
/*
newobj = PyString_FromStringAndSize(NULL, len);
if (newobj == NULL)
return NULL;
s_new = PyString_AsString(newobj);
*/
if (0 < len) {
int c = Py_CHARMASK(*s++);
if (Py_ISLOWER(c))
*result = Py_TOUPPER(c);
else
*result = c;
result++;
}
for (i = 1; i < len; i++) {
int c = Py_CHARMASK(*s++);
if (Py_ISUPPER(c))
*result = Py_TOLOWER(c);
else
*result = c;
result++;
}
}
PyDoc_STRVAR_shared(_Py_swapcase__doc__,
"B.swapcase() -> copy of B\n\
\n\
Return a copy of B with uppercase ASCII characters converted\n\
to lowercase ASCII and vice versa.");
void
_Py_bytes_swapcase(char *result, char *s, Py_ssize_t len)
{
Py_ssize_t i;
/*
newobj = PyString_FromStringAndSize(NULL, len);
if (newobj == NULL)
return NULL;
s_new = PyString_AsString(newobj);
*/
for (i = 0; i < len; i++) {
int c = Py_CHARMASK(*s++);
if (Py_ISLOWER(c)) {
*result = Py_TOUPPER(c);
}
else if (Py_ISUPPER(c)) {
*result = Py_TOLOWER(c);
}
else
*result = c;
result++;
}
}

View File

@ -0,0 +1,324 @@
/* Wrap void * pointers to be passed between C modules */
#include "Python.h"
/* Internal structure of PyCapsule */
typedef struct {
PyObject_HEAD
void *pointer;
const char *name;
void *context;
PyCapsule_Destructor destructor;
} PyCapsule;
static int
_is_legal_capsule(PyCapsule *capsule, const char *invalid_capsule)
{
if (!capsule || !PyCapsule_CheckExact(capsule) || capsule->pointer == NULL) {
PyErr_SetString(PyExc_ValueError, invalid_capsule);
return 0;
}
return 1;
}
#define is_legal_capsule(capsule, name) \
(_is_legal_capsule(capsule, \
name " called with invalid PyCapsule object"))
static int
name_matches(const char *name1, const char *name2) {
/* if either is NULL, */
if (!name1 || !name2) {
/* they're only the same if they're both NULL. */
return name1 == name2;
}
return !strcmp(name1, name2);
}
PyObject *
PyCapsule_New(void *pointer, const char *name, PyCapsule_Destructor destructor)
{
PyCapsule *capsule;
if (!pointer) {
PyErr_SetString(PyExc_ValueError, "PyCapsule_New called with null pointer");
return NULL;
}
capsule = PyObject_NEW(PyCapsule, &PyCapsule_Type);
if (capsule == NULL) {
return NULL;
}
capsule->pointer = pointer;
capsule->name = name;
capsule->context = NULL;
capsule->destructor = destructor;
return (PyObject *)capsule;
}
int
PyCapsule_IsValid(PyObject *o, const char *name)
{
PyCapsule *capsule = (PyCapsule *)o;
return (capsule != NULL &&
PyCapsule_CheckExact(capsule) &&
capsule->pointer != NULL &&
name_matches(capsule->name, name));
}
void *
PyCapsule_GetPointer(PyObject *o, const char *name)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_GetPointer")) {
return NULL;
}
if (!name_matches(name, capsule->name)) {
PyErr_SetString(PyExc_ValueError, "PyCapsule_GetPointer called with incorrect name");
return NULL;
}
return capsule->pointer;
}
const char *
PyCapsule_GetName(PyObject *o)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_GetName")) {
return NULL;
}
return capsule->name;
}
PyCapsule_Destructor
PyCapsule_GetDestructor(PyObject *o)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_GetDestructor")) {
return NULL;
}
return capsule->destructor;
}
void *
PyCapsule_GetContext(PyObject *o)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_GetContext")) {
return NULL;
}
return capsule->context;
}
int
PyCapsule_SetPointer(PyObject *o, void *pointer)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!pointer) {
PyErr_SetString(PyExc_ValueError, "PyCapsule_SetPointer called with null pointer");
return -1;
}
if (!is_legal_capsule(capsule, "PyCapsule_SetPointer")) {
return -1;
}
capsule->pointer = pointer;
return 0;
}
int
PyCapsule_SetName(PyObject *o, const char *name)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_SetName")) {
return -1;
}
capsule->name = name;
return 0;
}
int
PyCapsule_SetDestructor(PyObject *o, PyCapsule_Destructor destructor)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_SetDestructor")) {
return -1;
}
capsule->destructor = destructor;
return 0;
}
int
PyCapsule_SetContext(PyObject *o, void *context)
{
PyCapsule *capsule = (PyCapsule *)o;
if (!is_legal_capsule(capsule, "PyCapsule_SetContext")) {
return -1;
}
capsule->context = context;
return 0;
}
void *
PyCapsule_Import(const char *name, int no_block)
{
PyObject *object = NULL;
void *return_value = NULL;
char *trace;
size_t name_length = (strlen(name) + 1) * sizeof(char);
char *name_dup = (char *)PyMem_MALLOC(name_length);
if (!name_dup) {
return NULL;
}
memcpy(name_dup, name, name_length);
trace = name_dup;
while (trace) {
char *dot = strchr(trace, '.');
if (dot) {
*dot++ = '\0';
}
if (object == NULL) {
if (no_block) {
object = PyImport_ImportModuleNoBlock(trace);
} else {
object = PyImport_ImportModule(trace);
if (!object) {
PyErr_Format(PyExc_ImportError, "PyCapsule_Import could not import module \"%s\"", trace);
}
}
} else {
PyObject *object2 = PyObject_GetAttrString(object, trace);
Py_DECREF(object);
object = object2;
}
if (!object) {
goto EXIT;
}
trace = dot;
}
/* compare attribute name to module.name by hand */
if (PyCapsule_IsValid(object, name)) {
PyCapsule *capsule = (PyCapsule *)object;
return_value = capsule->pointer;
} else {
PyErr_Format(PyExc_AttributeError,
"PyCapsule_Import \"%s\" is not valid",
name);
}
EXIT:
Py_XDECREF(object);
if (name_dup) {
PyMem_FREE(name_dup);
}
return return_value;
}
static void
capsule_dealloc(PyObject *o)
{
PyCapsule *capsule = (PyCapsule *)o;
if (capsule->destructor) {
capsule->destructor(o);
}
PyObject_DEL(o);
}
static PyObject *
capsule_repr(PyObject *o)
{
PyCapsule *capsule = (PyCapsule *)o;
const char *name;
const char *quote;
if (capsule->name) {
quote = "\"";
name = capsule->name;
} else {
quote = "";
name = "NULL";
}
return PyString_FromFormat("<capsule object %s%s%s at %p>",
quote, name, quote, capsule);
}
PyDoc_STRVAR(PyCapsule_Type__doc__,
"Capsule objects let you wrap a C \"void *\" pointer in a Python\n\
object. They're a way of passing data through the Python interpreter\n\
without creating your own custom type.\n\
\n\
Capsules are used for communication between extension modules.\n\
They provide a way for an extension module to export a C interface\n\
to other extension modules, so that extension modules can use the\n\
Python import mechanism to link to one another.\n\
");
PyTypeObject PyCapsule_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"PyCapsule", /*tp_name*/
sizeof(PyCapsule), /*tp_basicsize*/
0, /*tp_itemsize*/
/* methods */
capsule_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_reserved*/
capsule_repr, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash*/
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
0, /*tp_flags*/
PyCapsule_Type__doc__ /*tp_doc*/
};

View File

@ -0,0 +1,145 @@
/* Cell object implementation */
#include "Python.h"
PyObject *
PyCell_New(PyObject *obj)
{
PyCellObject *op;
op = (PyCellObject *)PyObject_GC_New(PyCellObject, &PyCell_Type);
if (op == NULL)
return NULL;
op->ob_ref = obj;
Py_XINCREF(obj);
_PyObject_GC_TRACK(op);
return (PyObject *)op;
}
PyObject *
PyCell_Get(PyObject *op)
{
if (!PyCell_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
Py_XINCREF(((PyCellObject*)op)->ob_ref);
return PyCell_GET(op);
}
int
PyCell_Set(PyObject *op, PyObject *obj)
{
PyObject* oldobj;
if (!PyCell_Check(op)) {
PyErr_BadInternalCall();
return -1;
}
oldobj = PyCell_GET(op);
Py_XINCREF(obj);
PyCell_SET(op, obj);
Py_XDECREF(oldobj);
return 0;
}
static void
cell_dealloc(PyCellObject *op)
{
_PyObject_GC_UNTRACK(op);
Py_XDECREF(op->ob_ref);
PyObject_GC_Del(op);
}
static int
cell_compare(PyCellObject *a, PyCellObject *b)
{
/* Py3K warning for comparisons */
if (PyErr_WarnPy3k("cell comparisons not supported in 3.x",
1) < 0) {
return -2;
}
if (a->ob_ref == NULL) {
if (b->ob_ref == NULL)
return 0;
return -1;
} else if (b->ob_ref == NULL)
return 1;
return PyObject_Compare(a->ob_ref, b->ob_ref);
}
static PyObject *
cell_repr(PyCellObject *op)
{
if (op->ob_ref == NULL)
return PyString_FromFormat("<cell at %p: empty>", op);
return PyString_FromFormat("<cell at %p: %.80s object at %p>",
op, op->ob_ref->ob_type->tp_name,
op->ob_ref);
}
static int
cell_traverse(PyCellObject *op, visitproc visit, void *arg)
{
Py_VISIT(op->ob_ref);
return 0;
}
static int
cell_clear(PyCellObject *op)
{
Py_CLEAR(op->ob_ref);
return 0;
}
static PyObject *
cell_get_contents(PyCellObject *op, void *closure)
{
if (op->ob_ref == NULL)
{
PyErr_SetString(PyExc_ValueError, "Cell is empty");
return NULL;
}
Py_INCREF(op->ob_ref);
return op->ob_ref;
}
static PyGetSetDef cell_getsetlist[] = {
{"cell_contents", (getter)cell_get_contents, NULL},
{NULL} /* sentinel */
};
PyTypeObject PyCell_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"cell",
sizeof(PyCellObject),
0,
(destructor)cell_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
(cmpfunc)cell_compare, /* tp_compare */
(reprfunc)cell_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
0, /* tp_doc */
(traverseproc)cell_traverse, /* tp_traverse */
(inquiry)cell_clear, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
cell_getsetlist, /* tp_getset */
};

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,172 @@
/* Wrap void* pointers to be passed between C modules */
#include "Python.h"
/* Declarations for objects of type PyCObject */
typedef void (*destructor1)(void *);
typedef void (*destructor2)(void *, void*);
static int cobject_deprecation_warning(void)
{
return PyErr_WarnPy3k("CObject type is not supported in 3.x. "
"Please use capsule objects instead.", 1);
}
PyObject *
PyCObject_FromVoidPtr(void *cobj, void (*destr)(void *))
{
PyCObject *self;
if (cobject_deprecation_warning()) {
return NULL;
}
self = PyObject_NEW(PyCObject, &PyCObject_Type);
if (self == NULL)
return NULL;
self->cobject=cobj;
self->destructor=destr;
self->desc=NULL;
return (PyObject *)self;
}
PyObject *
PyCObject_FromVoidPtrAndDesc(void *cobj, void *desc,
void (*destr)(void *, void *))
{
PyCObject *self;
if (cobject_deprecation_warning()) {
return NULL;
}
if (!desc) {
PyErr_SetString(PyExc_TypeError,
"PyCObject_FromVoidPtrAndDesc called with null"
" description");
return NULL;
}
self = PyObject_NEW(PyCObject, &PyCObject_Type);
if (self == NULL)
return NULL;
self->cobject = cobj;
self->destructor = (destructor1)destr;
self->desc = desc;
return (PyObject *)self;
}
void *
PyCObject_AsVoidPtr(PyObject *self)
{
if (self) {
if (PyCapsule_CheckExact(self)) {
const char *name = PyCapsule_GetName(self);
return (void *)PyCapsule_GetPointer(self, name);
}
if (self->ob_type == &PyCObject_Type)
return ((PyCObject *)self)->cobject;
PyErr_SetString(PyExc_TypeError,
"PyCObject_AsVoidPtr with non-C-object");
}
if (!PyErr_Occurred())
PyErr_SetString(PyExc_TypeError,
"PyCObject_AsVoidPtr called with null pointer");
return NULL;
}
void *
PyCObject_GetDesc(PyObject *self)
{
if (self) {
if (self->ob_type == &PyCObject_Type)
return ((PyCObject *)self)->desc;
PyErr_SetString(PyExc_TypeError,
"PyCObject_GetDesc with non-C-object");
}
if (!PyErr_Occurred())
PyErr_SetString(PyExc_TypeError,
"PyCObject_GetDesc called with null pointer");
return NULL;
}
void *
PyCObject_Import(char *module_name, char *name)
{
PyObject *m, *c;
void *r = NULL;
if ((m = PyImport_ImportModule(module_name))) {
if ((c = PyObject_GetAttrString(m,name))) {
r = PyCObject_AsVoidPtr(c);
Py_DECREF(c);
}
Py_DECREF(m);
}
return r;
}
int
PyCObject_SetVoidPtr(PyObject *self, void *cobj)
{
PyCObject* cself = (PyCObject*)self;
if (cself == NULL || !PyCObject_Check(cself) ||
cself->destructor != NULL) {
PyErr_SetString(PyExc_TypeError,
"Invalid call to PyCObject_SetVoidPtr");
return 0;
}
cself->cobject = cobj;
return 1;
}
static void
PyCObject_dealloc(PyCObject *self)
{
if (self->destructor) {
if(self->desc)
((destructor2)(self->destructor))(self->cobject, self->desc);
else
(self->destructor)(self->cobject);
}
PyObject_DEL(self);
}
PyDoc_STRVAR(PyCObject_Type__doc__,
"C objects to be exported from one extension module to another\n\
\n\
C objects are used for communication between extension modules. They\n\
provide a way for an extension module to export a C interface to other\n\
extension modules, so that extension modules can use the Python import\n\
mechanism to link to one another.");
PyTypeObject PyCObject_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"PyCObject", /*tp_name*/
sizeof(PyCObject), /*tp_basicsize*/
0, /*tp_itemsize*/
/* methods */
(destructor)PyCObject_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash*/
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
0, /*tp_flags*/
PyCObject_Type__doc__ /*tp_doc*/
};

View File

@ -0,0 +1,581 @@
#include "Python.h"
#include "code.h"
#include "structmember.h"
#define NAME_CHARS \
"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz"
/* all_name_chars(s): true iff all chars in s are valid NAME_CHARS */
static int
all_name_chars(unsigned char *s)
{
static char ok_name_char[256];
static unsigned char *name_chars = (unsigned char *)NAME_CHARS;
if (ok_name_char[*name_chars] == 0) {
unsigned char *p;
for (p = name_chars; *p; p++)
ok_name_char[*p] = 1;
}
while (*s) {
if (ok_name_char[*s++] == 0)
return 0;
}
return 1;
}
static void
intern_strings(PyObject *tuple)
{
Py_ssize_t i;
for (i = PyTuple_GET_SIZE(tuple); --i >= 0; ) {
PyObject *v = PyTuple_GET_ITEM(tuple, i);
if (v == NULL || !PyString_CheckExact(v)) {
Py_FatalError("non-string found in code slot");
}
PyString_InternInPlace(&PyTuple_GET_ITEM(tuple, i));
}
}
PyCodeObject *
PyCode_New(int argcount, int nlocals, int stacksize, int flags,
PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *freevars, PyObject *cellvars,
PyObject *filename, PyObject *name, int firstlineno,
PyObject *lnotab)
{
PyCodeObject *co;
Py_ssize_t i;
/* Check argument types */
if (argcount < 0 || nlocals < 0 ||
code == NULL ||
consts == NULL || !PyTuple_Check(consts) ||
names == NULL || !PyTuple_Check(names) ||
varnames == NULL || !PyTuple_Check(varnames) ||
freevars == NULL || !PyTuple_Check(freevars) ||
cellvars == NULL || !PyTuple_Check(cellvars) ||
name == NULL || !PyString_Check(name) ||
filename == NULL || !PyString_Check(filename) ||
lnotab == NULL || !PyString_Check(lnotab) ||
!PyObject_CheckReadBuffer(code)) {
PyErr_BadInternalCall();
return NULL;
}
intern_strings(names);
intern_strings(varnames);
intern_strings(freevars);
intern_strings(cellvars);
/* Intern selected string constants */
for (i = PyTuple_Size(consts); --i >= 0; ) {
PyObject *v = PyTuple_GetItem(consts, i);
if (!PyString_Check(v))
continue;
if (!all_name_chars((unsigned char *)PyString_AS_STRING(v)))
continue;
PyString_InternInPlace(&PyTuple_GET_ITEM(consts, i));
}
co = PyObject_NEW(PyCodeObject, &PyCode_Type);
if (co != NULL) {
co->co_argcount = argcount;
co->co_nlocals = nlocals;
co->co_stacksize = stacksize;
co->co_flags = flags;
Py_INCREF(code);
co->co_code = code;
Py_INCREF(consts);
co->co_consts = consts;
Py_INCREF(names);
co->co_names = names;
Py_INCREF(varnames);
co->co_varnames = varnames;
Py_INCREF(freevars);
co->co_freevars = freevars;
Py_INCREF(cellvars);
co->co_cellvars = cellvars;
Py_INCREF(filename);
co->co_filename = filename;
Py_INCREF(name);
co->co_name = name;
co->co_firstlineno = firstlineno;
Py_INCREF(lnotab);
co->co_lnotab = lnotab;
co->co_zombieframe = NULL;
co->co_weakreflist = NULL;
}
return co;
}
PyCodeObject *
PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno)
{
static PyObject *emptystring = NULL;
static PyObject *nulltuple = NULL;
PyObject *filename_ob = NULL;
PyObject *funcname_ob = NULL;
PyCodeObject *result = NULL;
if (emptystring == NULL) {
emptystring = PyString_FromString("");
if (emptystring == NULL)
goto failed;
}
if (nulltuple == NULL) {
nulltuple = PyTuple_New(0);
if (nulltuple == NULL)
goto failed;
}
funcname_ob = PyString_FromString(funcname);
if (funcname_ob == NULL)
goto failed;
filename_ob = PyString_FromString(filename);
if (filename_ob == NULL)
goto failed;
result = PyCode_New(0, /* argcount */
0, /* nlocals */
0, /* stacksize */
0, /* flags */
emptystring, /* code */
nulltuple, /* consts */
nulltuple, /* names */
nulltuple, /* varnames */
nulltuple, /* freevars */
nulltuple, /* cellvars */
filename_ob, /* filename */
funcname_ob, /* name */
firstlineno, /* firstlineno */
emptystring /* lnotab */
);
failed:
Py_XDECREF(funcname_ob);
Py_XDECREF(filename_ob);
return result;
}
#define OFF(x) offsetof(PyCodeObject, x)
static PyMemberDef code_memberlist[] = {
{"co_argcount", T_INT, OFF(co_argcount), READONLY},
{"co_nlocals", T_INT, OFF(co_nlocals), READONLY},
{"co_stacksize",T_INT, OFF(co_stacksize), READONLY},
{"co_flags", T_INT, OFF(co_flags), READONLY},
{"co_code", T_OBJECT, OFF(co_code), READONLY},
{"co_consts", T_OBJECT, OFF(co_consts), READONLY},
{"co_names", T_OBJECT, OFF(co_names), READONLY},
{"co_varnames", T_OBJECT, OFF(co_varnames), READONLY},
{"co_freevars", T_OBJECT, OFF(co_freevars), READONLY},
{"co_cellvars", T_OBJECT, OFF(co_cellvars), READONLY},
{"co_filename", T_OBJECT, OFF(co_filename), READONLY},
{"co_name", T_OBJECT, OFF(co_name), READONLY},
{"co_firstlineno", T_INT, OFF(co_firstlineno), READONLY},
{"co_lnotab", T_OBJECT, OFF(co_lnotab), READONLY},
{NULL} /* Sentinel */
};
/* Helper for code_new: return a shallow copy of a tuple that is
guaranteed to contain exact strings, by converting string subclasses
to exact strings and complaining if a non-string is found. */
static PyObject*
validate_and_copy_tuple(PyObject *tup)
{
PyObject *newtuple;
PyObject *item;
Py_ssize_t i, len;
len = PyTuple_GET_SIZE(tup);
newtuple = PyTuple_New(len);
if (newtuple == NULL)
return NULL;
for (i = 0; i < len; i++) {
item = PyTuple_GET_ITEM(tup, i);
if (PyString_CheckExact(item)) {
Py_INCREF(item);
}
else if (!PyString_Check(item)) {
PyErr_Format(
PyExc_TypeError,
"name tuples must contain only "
"strings, not '%.500s'",
item->ob_type->tp_name);
Py_DECREF(newtuple);
return NULL;
}
else {
item = PyString_FromStringAndSize(
PyString_AS_STRING(item),
PyString_GET_SIZE(item));
if (item == NULL) {
Py_DECREF(newtuple);
return NULL;
}
}
PyTuple_SET_ITEM(newtuple, i, item);
}
return newtuple;
}
PyDoc_STRVAR(code_doc,
"code(argcount, nlocals, stacksize, flags, codestring, constants, names,\n\
varnames, filename, name, firstlineno, lnotab[, freevars[, cellvars]])\n\
\n\
Create a code object. Not for the faint of heart.");
static PyObject *
code_new(PyTypeObject *type, PyObject *args, PyObject *kw)
{
int argcount;
int nlocals;
int stacksize;
int flags;
PyObject *co = NULL;
PyObject *code;
PyObject *consts;
PyObject *names, *ournames = NULL;
PyObject *varnames, *ourvarnames = NULL;
PyObject *freevars = NULL, *ourfreevars = NULL;
PyObject *cellvars = NULL, *ourcellvars = NULL;
PyObject *filename;
PyObject *name;
int firstlineno;
PyObject *lnotab;
if (!PyArg_ParseTuple(args, "iiiiSO!O!O!SSiS|O!O!:code",
&argcount, &nlocals, &stacksize, &flags,
&code,
&PyTuple_Type, &consts,
&PyTuple_Type, &names,
&PyTuple_Type, &varnames,
&filename, &name,
&firstlineno, &lnotab,
&PyTuple_Type, &freevars,
&PyTuple_Type, &cellvars))
return NULL;
if (argcount < 0) {
PyErr_SetString(
PyExc_ValueError,
"code: argcount must not be negative");
goto cleanup;
}
if (nlocals < 0) {
PyErr_SetString(
PyExc_ValueError,
"code: nlocals must not be negative");
goto cleanup;
}
ournames = validate_and_copy_tuple(names);
if (ournames == NULL)
goto cleanup;
ourvarnames = validate_and_copy_tuple(varnames);
if (ourvarnames == NULL)
goto cleanup;
if (freevars)
ourfreevars = validate_and_copy_tuple(freevars);
else
ourfreevars = PyTuple_New(0);
if (ourfreevars == NULL)
goto cleanup;
if (cellvars)
ourcellvars = validate_and_copy_tuple(cellvars);
else
ourcellvars = PyTuple_New(0);
if (ourcellvars == NULL)
goto cleanup;
co = (PyObject *)PyCode_New(argcount, nlocals, stacksize, flags,
code, consts, ournames, ourvarnames,
ourfreevars, ourcellvars, filename,
name, firstlineno, lnotab);
cleanup:
Py_XDECREF(ournames);
Py_XDECREF(ourvarnames);
Py_XDECREF(ourfreevars);
Py_XDECREF(ourcellvars);
return co;
}
static void
code_dealloc(PyCodeObject *co)
{
Py_XDECREF(co->co_code);
Py_XDECREF(co->co_consts);
Py_XDECREF(co->co_names);
Py_XDECREF(co->co_varnames);
Py_XDECREF(co->co_freevars);
Py_XDECREF(co->co_cellvars);
Py_XDECREF(co->co_filename);
Py_XDECREF(co->co_name);
Py_XDECREF(co->co_lnotab);
if (co->co_zombieframe != NULL)
PyObject_GC_Del(co->co_zombieframe);
if (co->co_weakreflist != NULL)
PyObject_ClearWeakRefs((PyObject*)co);
PyObject_DEL(co);
}
static PyObject *
code_repr(PyCodeObject *co)
{
char buf[500];
int lineno = -1;
char *filename = "???";
char *name = "???";
if (co->co_firstlineno != 0)
lineno = co->co_firstlineno;
if (co->co_filename && PyString_Check(co->co_filename))
filename = PyString_AS_STRING(co->co_filename);
if (co->co_name && PyString_Check(co->co_name))
name = PyString_AS_STRING(co->co_name);
PyOS_snprintf(buf, sizeof(buf),
"<code object %.100s at %p, file \"%.300s\", line %d>",
name, co, filename, lineno);
return PyString_FromString(buf);
}
static int
code_compare(PyCodeObject *co, PyCodeObject *cp)
{
int cmp;
cmp = PyObject_Compare(co->co_name, cp->co_name);
if (cmp) return cmp;
cmp = co->co_argcount - cp->co_argcount;
if (cmp) goto normalize;
cmp = co->co_nlocals - cp->co_nlocals;
if (cmp) goto normalize;
cmp = co->co_flags - cp->co_flags;
if (cmp) goto normalize;
cmp = co->co_firstlineno - cp->co_firstlineno;
if (cmp) goto normalize;
cmp = PyObject_Compare(co->co_code, cp->co_code);
if (cmp) return cmp;
cmp = PyObject_Compare(co->co_consts, cp->co_consts);
if (cmp) return cmp;
cmp = PyObject_Compare(co->co_names, cp->co_names);
if (cmp) return cmp;
cmp = PyObject_Compare(co->co_varnames, cp->co_varnames);
if (cmp) return cmp;
cmp = PyObject_Compare(co->co_freevars, cp->co_freevars);
if (cmp) return cmp;
cmp = PyObject_Compare(co->co_cellvars, cp->co_cellvars);
return cmp;
normalize:
if (cmp > 0)
return 1;
else if (cmp < 0)
return -1;
else
return 0;
}
static PyObject *
code_richcompare(PyObject *self, PyObject *other, int op)
{
PyCodeObject *co, *cp;
int eq;
PyObject *res;
if ((op != Py_EQ && op != Py_NE) ||
!PyCode_Check(self) ||
!PyCode_Check(other)) {
/* Py3K warning if types are not equal and comparison
isn't == or != */
if (PyErr_WarnPy3k("code inequality comparisons not supported "
"in 3.x", 1) < 0) {
return NULL;
}
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
co = (PyCodeObject *)self;
cp = (PyCodeObject *)other;
eq = PyObject_RichCompareBool(co->co_name, cp->co_name, Py_EQ);
if (eq <= 0) goto unequal;
eq = co->co_argcount == cp->co_argcount;
if (!eq) goto unequal;
eq = co->co_nlocals == cp->co_nlocals;
if (!eq) goto unequal;
eq = co->co_flags == cp->co_flags;
if (!eq) goto unequal;
eq = co->co_firstlineno == cp->co_firstlineno;
if (!eq) goto unequal;
eq = PyObject_RichCompareBool(co->co_code, cp->co_code, Py_EQ);
if (eq <= 0) goto unequal;
eq = PyObject_RichCompareBool(co->co_consts, cp->co_consts, Py_EQ);
if (eq <= 0) goto unequal;
eq = PyObject_RichCompareBool(co->co_names, cp->co_names, Py_EQ);
if (eq <= 0) goto unequal;
eq = PyObject_RichCompareBool(co->co_varnames, cp->co_varnames, Py_EQ);
if (eq <= 0) goto unequal;
eq = PyObject_RichCompareBool(co->co_freevars, cp->co_freevars, Py_EQ);
if (eq <= 0) goto unequal;
eq = PyObject_RichCompareBool(co->co_cellvars, cp->co_cellvars, Py_EQ);
if (eq <= 0) goto unequal;
if (op == Py_EQ)
res = Py_True;
else
res = Py_False;
goto done;
unequal:
if (eq < 0)
return NULL;
if (op == Py_NE)
res = Py_True;
else
res = Py_False;
done:
Py_INCREF(res);
return res;
}
static long
code_hash(PyCodeObject *co)
{
long h, h0, h1, h2, h3, h4, h5, h6;
h0 = PyObject_Hash(co->co_name);
if (h0 == -1) return -1;
h1 = PyObject_Hash(co->co_code);
if (h1 == -1) return -1;
h2 = PyObject_Hash(co->co_consts);
if (h2 == -1) return -1;
h3 = PyObject_Hash(co->co_names);
if (h3 == -1) return -1;
h4 = PyObject_Hash(co->co_varnames);
if (h4 == -1) return -1;
h5 = PyObject_Hash(co->co_freevars);
if (h5 == -1) return -1;
h6 = PyObject_Hash(co->co_cellvars);
if (h6 == -1) return -1;
h = h0 ^ h1 ^ h2 ^ h3 ^ h4 ^ h5 ^ h6 ^
co->co_argcount ^ co->co_nlocals ^ co->co_flags;
if (h == -1) h = -2;
return h;
}
/* XXX code objects need to participate in GC? */
PyTypeObject PyCode_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"code",
sizeof(PyCodeObject),
0,
(destructor)code_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
(cmpfunc)code_compare, /* tp_compare */
(reprfunc)code_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
(hashfunc)code_hash, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
code_doc, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
code_richcompare, /* tp_richcompare */
offsetof(PyCodeObject, co_weakreflist), /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
code_memberlist, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
code_new, /* tp_new */
};
/* Use co_lnotab to compute the line number from a bytecode index, addrq. See
lnotab_notes.txt for the details of the lnotab representation.
*/
int
PyCode_Addr2Line(PyCodeObject *co, int addrq)
{
int size = PyString_Size(co->co_lnotab) / 2;
unsigned char *p = (unsigned char*)PyString_AsString(co->co_lnotab);
int line = co->co_firstlineno;
int addr = 0;
while (--size >= 0) {
addr += *p++;
if (addr > addrq)
break;
line += *p++;
}
return line;
}
/* Update *bounds to describe the first and one-past-the-last instructions in
the same line as lasti. Return the number of that line. */
int
_PyCode_CheckLineNumber(PyCodeObject* co, int lasti, PyAddrPair *bounds)
{
int size, addr, line;
unsigned char* p;
p = (unsigned char*)PyString_AS_STRING(co->co_lnotab);
size = PyString_GET_SIZE(co->co_lnotab) / 2;
addr = 0;
line = co->co_firstlineno;
assert(line > 0);
/* possible optimization: if f->f_lasti == instr_ub
(likely to be a common case) then we already know
instr_lb -- if we stored the matching value of p
somwhere we could skip the first while loop. */
/* See lnotab_notes.txt for the description of
co_lnotab. A point to remember: increments to p
come in (addr, line) pairs. */
bounds->ap_lower = 0;
while (size > 0) {
if (addr + *p > lasti)
break;
addr += *p++;
if (*p)
bounds->ap_lower = addr;
line += *p++;
--size;
}
if (size > 0) {
while (--size >= 0) {
addr += *p++;
if (*p++)
break;
}
bounds->ap_upper = addr;
}
else {
bounds->ap_upper = INT_MAX;
}
return line;
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,270 @@
NOTES ON OPTIMIZING DICTIONARIES
================================
Principal Use Cases for Dictionaries
------------------------------------
Passing keyword arguments
Typically, one read and one write for 1 to 3 elements.
Occurs frequently in normal python code.
Class method lookup
Dictionaries vary in size with 8 to 16 elements being common.
Usually written once with many lookups.
When base classes are used, there are many failed lookups
followed by a lookup in a base class.
Instance attribute lookup and Global variables
Dictionaries vary in size. 4 to 10 elements are common.
Both reads and writes are common.
Builtins
Frequent reads. Almost never written.
Size 126 interned strings (as of Py2.3b1).
A few keys are accessed much more frequently than others.
Uniquification
Dictionaries of any size. Bulk of work is in creation.
Repeated writes to a smaller set of keys.
Single read of each key.
Some use cases have two consecutive accesses to the same key.
* Removing duplicates from a sequence.
dict.fromkeys(seqn).keys()
* Counting elements in a sequence.
for e in seqn:
d[e] = d.get(e,0) + 1
* Accumulating references in a dictionary of lists:
for pagenumber, page in enumerate(pages):
for word in page:
d.setdefault(word, []).append(pagenumber)
Note, the second example is a use case characterized by a get and set
to the same key. There are similar use cases with a __contains__
followed by a get, set, or del to the same key. Part of the
justification for d.setdefault is combining the two lookups into one.
Membership Testing
Dictionaries of any size. Created once and then rarely changes.
Single write to each key.
Many calls to __contains__() or has_key().
Similar access patterns occur with replacement dictionaries
such as with the % formatting operator.
Dynamic Mappings
Characterized by deletions interspersed with adds and replacements.
Performance benefits greatly from the re-use of dummy entries.
Data Layout (assuming a 32-bit box with 64 bytes per cache line)
----------------------------------------------------------------
Smalldicts (8 entries) are attached to the dictobject structure
and the whole group nearly fills two consecutive cache lines.
Larger dicts use the first half of the dictobject structure (one cache
line) and a separate, continuous block of entries (at 12 bytes each
for a total of 5.333 entries per cache line).
Tunable Dictionary Parameters
-----------------------------
* PyDict_MINSIZE. Currently set to 8.
Must be a power of two. New dicts have to zero-out every cell.
Each additional 8 consumes 1.5 cache lines. Increasing improves
the sparseness of small dictionaries but costs time to read in
the additional cache lines if they are not already in cache.
That case is common when keyword arguments are passed.
* Maximum dictionary load in PyDict_SetItem. Currently set to 2/3.
Increasing this ratio makes dictionaries more dense resulting
in more collisions. Decreasing it improves sparseness at the
expense of spreading entries over more cache lines and at the
cost of total memory consumed.
The load test occurs in highly time sensitive code. Efforts
to make the test more complex (for example, varying the load
for different sizes) have degraded performance.
* Growth rate upon hitting maximum load. Currently set to *2.
Raising this to *4 results in half the number of resizes,
less effort to resize, better sparseness for some (but not
all dict sizes), and potentially doubles memory consumption
depending on the size of the dictionary. Setting to *4
eliminates every other resize step.
* Maximum sparseness (minimum dictionary load). What percentage
of entries can be unused before the dictionary shrinks to
free up memory and speed up iteration? (The current CPython
code does not represent this parameter directly.)
* Shrinkage rate upon exceeding maximum sparseness. The current
CPython code never even checks sparseness when deleting a
key. When a new key is added, it resizes based on the number
of active keys, so that the addition may trigger shrinkage
rather than growth.
Tune-ups should be measured across a broad range of applications and
use cases. A change to any parameter will help in some situations and
hurt in others. The key is to find settings that help the most common
cases and do the least damage to the less common cases. Results will
vary dramatically depending on the exact number of keys, whether the
keys are all strings, whether reads or writes dominate, the exact
hash values of the keys (some sets of values have fewer collisions than
others). Any one test or benchmark is likely to prove misleading.
While making a dictionary more sparse reduces collisions, it impairs
iteration and key listing. Those methods loop over every potential
entry. Doubling the size of dictionary results in twice as many
non-overlapping memory accesses for keys(), items(), values(),
__iter__(), iterkeys(), iteritems(), itervalues(), and update().
Also, every dictionary iterates at least twice, once for the memset()
when it is created and once by dealloc().
Dictionary operations involving only a single key can be O(1) unless
resizing is possible. By checking for a resize only when the
dictionary can grow (and may *require* resizing), other operations
remain O(1), and the odds of resize thrashing or memory fragmentation
are reduced. In particular, an algorithm that empties a dictionary
by repeatedly invoking .pop will see no resizing, which might
not be necessary at all because the dictionary is eventually
discarded entirely.
Results of Cache Locality Experiments
-------------------------------------
When an entry is retrieved from memory, 4.333 adjacent entries are also
retrieved into a cache line. Since accessing items in cache is *much*
cheaper than a cache miss, an enticing idea is to probe the adjacent
entries as a first step in collision resolution. Unfortunately, the
introduction of any regularity into collision searches results in more
collisions than the current random chaining approach.
Exploiting cache locality at the expense of additional collisions fails
to payoff when the entries are already loaded in cache (the expense
is paid with no compensating benefit). This occurs in small dictionaries
where the whole dictionary fits into a pair of cache lines. It also
occurs frequently in large dictionaries which have a common access pattern
where some keys are accessed much more frequently than others. The
more popular entries *and* their collision chains tend to remain in cache.
To exploit cache locality, change the collision resolution section
in lookdict() and lookdict_string(). Set i^=1 at the top of the
loop and move the i = (i << 2) + i + perturb + 1 to an unrolled
version of the loop.
This optimization strategy can be leveraged in several ways:
* If the dictionary is kept sparse (through the tunable parameters),
then the occurrence of additional collisions is lessened.
* If lookdict() and lookdict_string() are specialized for small dicts
and for largedicts, then the versions for large_dicts can be given
an alternate search strategy without increasing collisions in small dicts
which already have the maximum benefit of cache locality.
* If the use case for a dictionary is known to have a random key
access pattern (as opposed to a more common pattern with a Zipf's law
distribution), then there will be more benefit for large dictionaries
because any given key is no more likely than another to already be
in cache.
* In use cases with paired accesses to the same key, the second access
is always in cache and gets no benefit from efforts to further improve
cache locality.
Optimizing the Search of Small Dictionaries
-------------------------------------------
If lookdict() and lookdict_string() are specialized for smaller dictionaries,
then a custom search approach can be implemented that exploits the small
search space and cache locality.
* The simplest example is a linear search of contiguous entries. This is
simple to implement, guaranteed to terminate rapidly, never searches
the same entry twice, and precludes the need to check for dummy entries.
* A more advanced example is a self-organizing search so that the most
frequently accessed entries get probed first. The organization
adapts if the access pattern changes over time. Treaps are ideally
suited for self-organization with the most common entries at the
top of the heap and a rapid binary search pattern. Most probes and
results are all located at the top of the tree allowing them all to
be located in one or two cache lines.
* Also, small dictionaries may be made more dense, perhaps filling all
eight cells to take the maximum advantage of two cache lines.
Strategy Pattern
----------------
Consider allowing the user to set the tunable parameters or to select a
particular search method. Since some dictionary use cases have known
sizes and access patterns, the user may be able to provide useful hints.
1) For example, if membership testing or lookups dominate runtime and memory
is not at a premium, the user may benefit from setting the maximum load
ratio at 5% or 10% instead of the usual 66.7%. This will sharply
curtail the number of collisions but will increase iteration time.
The builtin namespace is a prime example of a dictionary that can
benefit from being highly sparse.
2) Dictionary creation time can be shortened in cases where the ultimate
size of the dictionary is known in advance. The dictionary can be
pre-sized so that no resize operations are required during creation.
Not only does this save resizes, but the key insertion will go
more quickly because the first half of the keys will be inserted into
a more sparse environment than before. The preconditions for this
strategy arise whenever a dictionary is created from a key or item
sequence and the number of *unique* keys is known.
3) If the key space is large and the access pattern is known to be random,
then search strategies exploiting cache locality can be fruitful.
The preconditions for this strategy arise in simulations and
numerical analysis.
4) If the keys are fixed and the access pattern strongly favors some of
the keys, then the entries can be stored contiguously and accessed
with a linear search or treap. This exploits knowledge of the data,
cache locality, and a simplified search routine. It also eliminates
the need to test for dummy entries on each probe. The preconditions
for this strategy arise in symbol tables and in the builtin dictionary.
Readonly Dictionaries
---------------------
Some dictionary use cases pass through a build stage and then move to a
more heavily exercised lookup stage with no further changes to the
dictionary.
An idea that emerged on python-dev is to be able to convert a dictionary
to a read-only state. This can help prevent programming errors and also
provide knowledge that can be exploited for lookup optimization.
The dictionary can be immediately rebuilt (eliminating dummy entries),
resized (to an appropriate level of sparseness), and the keys can be
jostled (to minimize collisions). The lookdict() routine can then
eliminate the test for dummy entries (saving about 1/4 of the time
spent in the collision resolution loop).
An additional possibility is to insert links into the empty spaces
so that dictionary iteration can proceed in len(d) steps instead of
(mp->mask + 1) steps. Alternatively, a separate tuple of keys can be
kept just for iteration.
Caching Lookups
---------------
The idea is to exploit key access patterns by anticipating future lookups
based on previous lookups.
The simplest incarnation is to save the most recently accessed entry.
This gives optimal performance for use cases where every get is followed
by a set or del to the same key.

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,381 @@
/* enumerate object */
#include "Python.h"
typedef struct {
PyObject_HEAD
Py_ssize_t en_index; /* current index of enumeration */
PyObject* en_sit; /* secondary iterator of enumeration */
PyObject* en_result; /* result tuple */
PyObject* en_longindex; /* index for sequences >= PY_SSIZE_T_MAX */
} enumobject;
static PyObject *
enum_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
enumobject *en;
PyObject *seq = NULL;
PyObject *start = NULL;
static char *kwlist[] = {"sequence", "start", 0};
if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:enumerate", kwlist,
&seq, &start))
return NULL;
en = (enumobject *)type->tp_alloc(type, 0);
if (en == NULL)
return NULL;
if (start != NULL) {
start = PyNumber_Index(start);
if (start == NULL) {
Py_DECREF(en);
return NULL;
}
assert(PyInt_Check(start) || PyLong_Check(start));
en->en_index = PyInt_AsSsize_t(start);
if (en->en_index == -1 && PyErr_Occurred()) {
PyErr_Clear();
en->en_index = PY_SSIZE_T_MAX;
en->en_longindex = start;
} else {
en->en_longindex = NULL;
Py_DECREF(start);
}
} else {
en->en_index = 0;
en->en_longindex = NULL;
}
en->en_sit = PyObject_GetIter(seq);
if (en->en_sit == NULL) {
Py_DECREF(en);
return NULL;
}
en->en_result = PyTuple_Pack(2, Py_None, Py_None);
if (en->en_result == NULL) {
Py_DECREF(en);
return NULL;
}
return (PyObject *)en;
}
static void
enum_dealloc(enumobject *en)
{
PyObject_GC_UnTrack(en);
Py_XDECREF(en->en_sit);
Py_XDECREF(en->en_result);
Py_XDECREF(en->en_longindex);
Py_TYPE(en)->tp_free(en);
}
static int
enum_traverse(enumobject *en, visitproc visit, void *arg)
{
Py_VISIT(en->en_sit);
Py_VISIT(en->en_result);
Py_VISIT(en->en_longindex);
return 0;
}
static PyObject *
enum_next_long(enumobject *en, PyObject* next_item)
{
static PyObject *one = NULL;
PyObject *result = en->en_result;
PyObject *next_index;
PyObject *stepped_up;
if (en->en_longindex == NULL) {
en->en_longindex = PyInt_FromSsize_t(PY_SSIZE_T_MAX);
if (en->en_longindex == NULL)
return NULL;
}
if (one == NULL) {
one = PyInt_FromLong(1);
if (one == NULL)
return NULL;
}
next_index = en->en_longindex;
assert(next_index != NULL);
stepped_up = PyNumber_Add(next_index, one);
if (stepped_up == NULL)
return NULL;
en->en_longindex = stepped_up;
if (result->ob_refcnt == 1) {
Py_INCREF(result);
Py_DECREF(PyTuple_GET_ITEM(result, 0));
Py_DECREF(PyTuple_GET_ITEM(result, 1));
} else {
result = PyTuple_New(2);
if (result == NULL) {
Py_DECREF(next_index);
Py_DECREF(next_item);
return NULL;
}
}
PyTuple_SET_ITEM(result, 0, next_index);
PyTuple_SET_ITEM(result, 1, next_item);
return result;
}
static PyObject *
enum_next(enumobject *en)
{
PyObject *next_index;
PyObject *next_item;
PyObject *result = en->en_result;
PyObject *it = en->en_sit;
next_item = (*Py_TYPE(it)->tp_iternext)(it);
if (next_item == NULL)
return NULL;
if (en->en_index == PY_SSIZE_T_MAX)
return enum_next_long(en, next_item);
next_index = PyInt_FromSsize_t(en->en_index);
if (next_index == NULL) {
Py_DECREF(next_item);
return NULL;
}
en->en_index++;
if (result->ob_refcnt == 1) {
Py_INCREF(result);
Py_DECREF(PyTuple_GET_ITEM(result, 0));
Py_DECREF(PyTuple_GET_ITEM(result, 1));
} else {
result = PyTuple_New(2);
if (result == NULL) {
Py_DECREF(next_index);
Py_DECREF(next_item);
return NULL;
}
}
PyTuple_SET_ITEM(result, 0, next_index);
PyTuple_SET_ITEM(result, 1, next_item);
return result;
}
PyDoc_STRVAR(enum_doc,
"enumerate(iterable[, start]) -> iterator for index, value of iterable\n"
"\n"
"Return an enumerate object. iterable must be another object that supports\n"
"iteration. The enumerate object yields pairs containing a count (from\n"
"start, which defaults to zero) and a value yielded by the iterable argument.\n"
"enumerate is useful for obtaining an indexed list:\n"
" (0, seq[0]), (1, seq[1]), (2, seq[2]), ...");
PyTypeObject PyEnum_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"enumerate", /* tp_name */
sizeof(enumobject), /* tp_basicsize */
0, /* tp_itemsize */
/* methods */
(destructor)enum_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
Py_TPFLAGS_BASETYPE, /* tp_flags */
enum_doc, /* tp_doc */
(traverseproc)enum_traverse, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
PyObject_SelfIter, /* tp_iter */
(iternextfunc)enum_next, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
PyType_GenericAlloc, /* tp_alloc */
enum_new, /* tp_new */
PyObject_GC_Del, /* tp_free */
};
/* Reversed Object ***************************************************************/
typedef struct {
PyObject_HEAD
Py_ssize_t index;
PyObject* seq;
} reversedobject;
static PyObject *
reversed_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Py_ssize_t n;
PyObject *seq, *reversed_meth;
static PyObject *reversed_cache = NULL;
reversedobject *ro;
if (type == &PyReversed_Type && !_PyArg_NoKeywords("reversed()", kwds))
return NULL;
if (!PyArg_UnpackTuple(args, "reversed", 1, 1, &seq) )
return NULL;
if (PyInstance_Check(seq)) {
reversed_meth = PyObject_GetAttrString(seq, "__reversed__");
if (reversed_meth == NULL) {
if (PyErr_ExceptionMatches(PyExc_AttributeError))
PyErr_Clear();
else
return NULL;
}
}
else {
reversed_meth = _PyObject_LookupSpecial(seq, "__reversed__",
&reversed_cache);
if (reversed_meth == NULL && PyErr_Occurred())
return NULL;
}
if (reversed_meth != NULL) {
PyObject *res = PyObject_CallFunctionObjArgs(reversed_meth, NULL);
Py_DECREF(reversed_meth);
return res;
}
if (!PySequence_Check(seq)) {
PyErr_SetString(PyExc_TypeError,
"argument to reversed() must be a sequence");
return NULL;
}
n = PySequence_Size(seq);
if (n == -1)
return NULL;
ro = (reversedobject *)type->tp_alloc(type, 0);
if (ro == NULL)
return NULL;
ro->index = n-1;
Py_INCREF(seq);
ro->seq = seq;
return (PyObject *)ro;
}
static void
reversed_dealloc(reversedobject *ro)
{
PyObject_GC_UnTrack(ro);
Py_XDECREF(ro->seq);
Py_TYPE(ro)->tp_free(ro);
}
static int
reversed_traverse(reversedobject *ro, visitproc visit, void *arg)
{
Py_VISIT(ro->seq);
return 0;
}
static PyObject *
reversed_next(reversedobject *ro)
{
PyObject *item;
Py_ssize_t index = ro->index;
if (index >= 0) {
item = PySequence_GetItem(ro->seq, index);
if (item != NULL) {
ro->index--;
return item;
}
if (PyErr_ExceptionMatches(PyExc_IndexError) ||
PyErr_ExceptionMatches(PyExc_StopIteration))
PyErr_Clear();
}
ro->index = -1;
Py_CLEAR(ro->seq);
return NULL;
}
PyDoc_STRVAR(reversed_doc,
"reversed(sequence) -> reverse iterator over values of the sequence\n"
"\n"
"Return a reverse iterator");
static PyObject *
reversed_len(reversedobject *ro)
{
Py_ssize_t position, seqsize;
if (ro->seq == NULL)
return PyInt_FromLong(0);
seqsize = PySequence_Size(ro->seq);
if (seqsize == -1)
return NULL;
position = ro->index + 1;
return PyInt_FromSsize_t((seqsize < position) ? 0 : position);
}
PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list(it)).");
static PyMethodDef reversediter_methods[] = {
{"__length_hint__", (PyCFunction)reversed_len, METH_NOARGS, length_hint_doc},
{NULL, NULL} /* sentinel */
};
PyTypeObject PyReversed_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"reversed", /* tp_name */
sizeof(reversedobject), /* tp_basicsize */
0, /* tp_itemsize */
/* methods */
(destructor)reversed_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
Py_TPFLAGS_BASETYPE, /* tp_flags */
reversed_doc, /* tp_doc */
(traverseproc)reversed_traverse,/* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
PyObject_SelfIter, /* tp_iter */
(iternextfunc)reversed_next, /* tp_iternext */
reversediter_methods, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
PyType_GenericAlloc, /* tp_alloc */
reversed_new, /* tp_new */
PyObject_GC_Del, /* tp_free */
};

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,984 @@
/* Frame object implementation */
#include "Python.h"
#include "code.h"
#include "frameobject.h"
#include "opcode.h"
#include "structmember.h"
#undef MIN
#undef MAX
#define MIN(a, b) ((a) < (b) ? (a) : (b))
#define MAX(a, b) ((a) > (b) ? (a) : (b))
#define OFF(x) offsetof(PyFrameObject, x)
static PyMemberDef frame_memberlist[] = {
{"f_back", T_OBJECT, OFF(f_back), RO},
{"f_code", T_OBJECT, OFF(f_code), RO},
{"f_builtins", T_OBJECT, OFF(f_builtins),RO},
{"f_globals", T_OBJECT, OFF(f_globals), RO},
{"f_lasti", T_INT, OFF(f_lasti), RO},
{NULL} /* Sentinel */
};
#define WARN_GET_SET(NAME) \
static PyObject * frame_get_ ## NAME(PyFrameObject *f) { \
if (PyErr_WarnPy3k(#NAME " has been removed in 3.x", 2) < 0) \
return NULL; \
if (f->NAME) { \
Py_INCREF(f->NAME); \
return f->NAME; \
} \
Py_RETURN_NONE; \
} \
static int frame_set_ ## NAME(PyFrameObject *f, PyObject *new) { \
if (PyErr_WarnPy3k(#NAME " has been removed in 3.x", 2) < 0) \
return -1; \
if (f->NAME) { \
Py_CLEAR(f->NAME); \
} \
if (new == Py_None) \
new = NULL; \
Py_XINCREF(new); \
f->NAME = new; \
return 0; \
}
WARN_GET_SET(f_exc_traceback)
WARN_GET_SET(f_exc_type)
WARN_GET_SET(f_exc_value)
static PyObject *
frame_getlocals(PyFrameObject *f, void *closure)
{
PyFrame_FastToLocals(f);
Py_INCREF(f->f_locals);
return f->f_locals;
}
int
PyFrame_GetLineNumber(PyFrameObject *f)
{
if (f->f_trace)
return f->f_lineno;
else
return PyCode_Addr2Line(f->f_code, f->f_lasti);
}
static PyObject *
frame_getlineno(PyFrameObject *f, void *closure)
{
return PyInt_FromLong(PyFrame_GetLineNumber(f));
}
/* Setter for f_lineno - you can set f_lineno from within a trace function in
* order to jump to a given line of code, subject to some restrictions. Most
* lines are OK to jump to because they don't make any assumptions about the
* state of the stack (obvious because you could remove the line and the code
* would still work without any stack errors), but there are some constructs
* that limit jumping:
*
* o Lines with an 'except' statement on them can't be jumped to, because
* they expect an exception to be on the top of the stack.
* o Lines that live in a 'finally' block can't be jumped from or to, since
* the END_FINALLY expects to clean up the stack after the 'try' block.
* o 'try'/'for'/'while' blocks can't be jumped into because the blockstack
* needs to be set up before their code runs, and for 'for' loops the
* iterator needs to be on the stack.
*/
static int
frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno)
{
int new_lineno = 0; /* The new value of f_lineno */
int new_lasti = 0; /* The new value of f_lasti */
int new_iblock = 0; /* The new value of f_iblock */
unsigned char *code = NULL; /* The bytecode for the frame... */
Py_ssize_t code_len = 0; /* ...and its length */
unsigned char *lnotab = NULL; /* Iterating over co_lnotab */
Py_ssize_t lnotab_len = 0; /* (ditto) */
int offset = 0; /* (ditto) */
int line = 0; /* (ditto) */
int addr = 0; /* (ditto) */
int min_addr = 0; /* Scanning the SETUPs and POPs */
int max_addr = 0; /* (ditto) */
int delta_iblock = 0; /* (ditto) */
int min_delta_iblock = 0; /* (ditto) */
int min_iblock = 0; /* (ditto) */
int f_lasti_setup_addr = 0; /* Policing no-jump-into-finally */
int new_lasti_setup_addr = 0; /* (ditto) */
int blockstack[CO_MAXBLOCKS]; /* Walking the 'finally' blocks */
int in_finally[CO_MAXBLOCKS]; /* (ditto) */
int blockstack_top = 0; /* (ditto) */
unsigned char setup_op = 0; /* (ditto) */
/* f_lineno must be an integer. */
if (!PyInt_Check(p_new_lineno)) {
PyErr_SetString(PyExc_ValueError,
"lineno must be an integer");
return -1;
}
/* You can only do this from within a trace function, not via
* _getframe or similar hackery. */
if (!f->f_trace)
{
PyErr_Format(PyExc_ValueError,
"f_lineno can only be set by a"
" line trace function");
return -1;
}
/* Fail if the line comes before the start of the code block. */
new_lineno = (int) PyInt_AsLong(p_new_lineno);
if (new_lineno < f->f_code->co_firstlineno) {
PyErr_Format(PyExc_ValueError,
"line %d comes before the current code block",
new_lineno);
return -1;
}
else if (new_lineno == f->f_code->co_firstlineno) {
new_lasti = 0;
new_lineno = f->f_code->co_firstlineno;
}
else {
/* Find the bytecode offset for the start of the given
* line, or the first code-owning line after it. */
char *tmp;
PyString_AsStringAndSize(f->f_code->co_lnotab,
&tmp, &lnotab_len);
lnotab = (unsigned char *) tmp;
addr = 0;
line = f->f_code->co_firstlineno;
new_lasti = -1;
for (offset = 0; offset < lnotab_len; offset += 2) {
addr += lnotab[offset];
line += lnotab[offset+1];
if (line >= new_lineno) {
new_lasti = addr;
new_lineno = line;
break;
}
}
}
/* If we didn't reach the requested line, return an error. */
if (new_lasti == -1) {
PyErr_Format(PyExc_ValueError,
"line %d comes after the current code block",
new_lineno);
return -1;
}
/* We're now ready to look at the bytecode. */
PyString_AsStringAndSize(f->f_code->co_code, (char **)&code, &code_len);
min_addr = MIN(new_lasti, f->f_lasti);
max_addr = MAX(new_lasti, f->f_lasti);
/* You can't jump onto a line with an 'except' statement on it -
* they expect to have an exception on the top of the stack, which
* won't be true if you jump to them. They always start with code
* that either pops the exception using POP_TOP (plain 'except:'
* lines do this) or duplicates the exception on the stack using
* DUP_TOP (if there's an exception type specified). See compile.c,
* 'com_try_except' for the full details. There aren't any other
* cases (AFAIK) where a line's code can start with DUP_TOP or
* POP_TOP, but if any ever appear, they'll be subject to the same
* restriction (but with a different error message). */
if (code[new_lasti] == DUP_TOP || code[new_lasti] == POP_TOP) {
PyErr_SetString(PyExc_ValueError,
"can't jump to 'except' line as there's no exception");
return -1;
}
/* You can't jump into or out of a 'finally' block because the 'try'
* block leaves something on the stack for the END_FINALLY to clean
* up. So we walk the bytecode, maintaining a simulated blockstack.
* When we reach the old or new address and it's in a 'finally' block
* we note the address of the corresponding SETUP_FINALLY. The jump
* is only legal if neither address is in a 'finally' block or
* they're both in the same one. 'blockstack' is a stack of the
* bytecode addresses of the SETUP_X opcodes, and 'in_finally' tracks
* whether we're in a 'finally' block at each blockstack level. */
f_lasti_setup_addr = -1;
new_lasti_setup_addr = -1;
memset(blockstack, '\0', sizeof(blockstack));
memset(in_finally, '\0', sizeof(in_finally));
blockstack_top = 0;
for (addr = 0; addr < code_len; addr++) {
unsigned char op = code[addr];
switch (op) {
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
blockstack[blockstack_top++] = addr;
in_finally[blockstack_top-1] = 0;
break;
case POP_BLOCK:
assert(blockstack_top > 0);
setup_op = code[blockstack[blockstack_top-1]];
if (setup_op == SETUP_FINALLY || setup_op == SETUP_WITH) {
in_finally[blockstack_top-1] = 1;
}
else {
blockstack_top--;
}
break;
case END_FINALLY:
/* Ignore END_FINALLYs for SETUP_EXCEPTs - they exist
* in the bytecode but don't correspond to an actual
* 'finally' block. (If blockstack_top is 0, we must
* be seeing such an END_FINALLY.) */
if (blockstack_top > 0) {
setup_op = code[blockstack[blockstack_top-1]];
if (setup_op == SETUP_FINALLY || setup_op == SETUP_WITH) {
blockstack_top--;
}
}
break;
}
/* For the addresses we're interested in, see whether they're
* within a 'finally' block and if so, remember the address
* of the SETUP_FINALLY. */
if (addr == new_lasti || addr == f->f_lasti) {
int i = 0;
int setup_addr = -1;
for (i = blockstack_top-1; i >= 0; i--) {
if (in_finally[i]) {
setup_addr = blockstack[i];
break;
}
}
if (setup_addr != -1) {
if (addr == new_lasti) {
new_lasti_setup_addr = setup_addr;
}
if (addr == f->f_lasti) {
f_lasti_setup_addr = setup_addr;
}
}
}
if (op >= HAVE_ARGUMENT) {
addr += 2;
}
}
/* Verify that the blockstack tracking code didn't get lost. */
assert(blockstack_top == 0);
/* After all that, are we jumping into / out of a 'finally' block? */
if (new_lasti_setup_addr != f_lasti_setup_addr) {
PyErr_SetString(PyExc_ValueError,
"can't jump into or out of a 'finally' block");
return -1;
}
/* Police block-jumping (you can't jump into the middle of a block)
* and ensure that the blockstack finishes up in a sensible state (by
* popping any blocks we're jumping out of). We look at all the
* blockstack operations between the current position and the new
* one, and keep track of how many blocks we drop out of on the way.
* By also keeping track of the lowest blockstack position we see, we
* can tell whether the jump goes into any blocks without coming out
* again - in that case we raise an exception below. */
delta_iblock = 0;
for (addr = min_addr; addr < max_addr; addr++) {
unsigned char op = code[addr];
switch (op) {
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
delta_iblock++;
break;
case POP_BLOCK:
delta_iblock--;
break;
}
min_delta_iblock = MIN(min_delta_iblock, delta_iblock);
if (op >= HAVE_ARGUMENT) {
addr += 2;
}
}
/* Derive the absolute iblock values from the deltas. */
min_iblock = f->f_iblock + min_delta_iblock;
if (new_lasti > f->f_lasti) {
/* Forwards jump. */
new_iblock = f->f_iblock + delta_iblock;
}
else {
/* Backwards jump. */
new_iblock = f->f_iblock - delta_iblock;
}
/* Are we jumping into a block? */
if (new_iblock > min_iblock) {
PyErr_SetString(PyExc_ValueError,
"can't jump into the middle of a block");
return -1;
}
/* Pop any blocks that we're jumping out of. */
while (f->f_iblock > new_iblock) {
PyTryBlock *b = &f->f_blockstack[--f->f_iblock];
while ((f->f_stacktop - f->f_valuestack) > b->b_level) {
PyObject *v = (*--f->f_stacktop);
Py_DECREF(v);
}
}
/* Finally set the new f_lineno and f_lasti and return OK. */
f->f_lineno = new_lineno;
f->f_lasti = new_lasti;
return 0;
}
static PyObject *
frame_gettrace(PyFrameObject *f, void *closure)
{
PyObject* trace = f->f_trace;
if (trace == NULL)
trace = Py_None;
Py_INCREF(trace);
return trace;
}
static int
frame_settrace(PyFrameObject *f, PyObject* v, void *closure)
{
PyObject* old_value;
/* We rely on f_lineno being accurate when f_trace is set. */
f->f_lineno = PyFrame_GetLineNumber(f);
old_value = f->f_trace;
Py_XINCREF(v);
f->f_trace = v;
Py_XDECREF(old_value);
return 0;
}
static PyObject *
frame_getrestricted(PyFrameObject *f, void *closure)
{
return PyBool_FromLong(PyFrame_IsRestricted(f));
}
static PyGetSetDef frame_getsetlist[] = {
{"f_locals", (getter)frame_getlocals, NULL, NULL},
{"f_lineno", (getter)frame_getlineno,
(setter)frame_setlineno, NULL},
{"f_trace", (getter)frame_gettrace, (setter)frame_settrace, NULL},
{"f_restricted",(getter)frame_getrestricted,NULL, NULL},
{"f_exc_traceback", (getter)frame_get_f_exc_traceback,
(setter)frame_set_f_exc_traceback, NULL},
{"f_exc_type", (getter)frame_get_f_exc_type,
(setter)frame_set_f_exc_type, NULL},
{"f_exc_value", (getter)frame_get_f_exc_value,
(setter)frame_set_f_exc_value, NULL},
{0}
};
/* Stack frames are allocated and deallocated at a considerable rate.
In an attempt to improve the speed of function calls, we:
1. Hold a single "zombie" frame on each code object. This retains
the allocated and initialised frame object from an invocation of
the code object. The zombie is reanimated the next time we need a
frame object for that code object. Doing this saves the malloc/
realloc required when using a free_list frame that isn't the
correct size. It also saves some field initialisation.
In zombie mode, no field of PyFrameObject holds a reference, but
the following fields are still valid:
* ob_type, ob_size, f_code, f_valuestack;
* f_locals, f_trace,
f_exc_type, f_exc_value, f_exc_traceback are NULL;
* f_localsplus does not require re-allocation and
the local variables in f_localsplus are NULL.
2. We also maintain a separate free list of stack frames (just like
integers are allocated in a special way -- see intobject.c). When
a stack frame is on the free list, only the following members have
a meaning:
ob_type == &Frametype
f_back next item on free list, or NULL
f_stacksize size of value stack
ob_size size of localsplus
Note that the value and block stacks are preserved -- this can save
another malloc() call or two (and two free() calls as well!).
Also note that, unlike for integers, each frame object is a
malloc'ed object in its own right -- it is only the actual calls to
malloc() that we are trying to save here, not the administration.
After all, while a typical program may make millions of calls, a
call depth of more than 20 or 30 is probably already exceptional
unless the program contains run-away recursion. I hope.
Later, PyFrame_MAXFREELIST was added to bound the # of frames saved on
free_list. Else programs creating lots of cyclic trash involving
frames could provoke free_list into growing without bound.
*/
static PyFrameObject *free_list = NULL;
static int numfree = 0; /* number of frames currently in free_list */
/* max value for numfree */
#define PyFrame_MAXFREELIST 200
static void
frame_dealloc(PyFrameObject *f)
{
PyObject **p, **valuestack;
PyCodeObject *co;
PyObject_GC_UnTrack(f);
Py_TRASHCAN_SAFE_BEGIN(f)
/* Kill all local variables */
valuestack = f->f_valuestack;
for (p = f->f_localsplus; p < valuestack; p++)
Py_CLEAR(*p);
/* Free stack */
if (f->f_stacktop != NULL) {
for (p = valuestack; p < f->f_stacktop; p++)
Py_XDECREF(*p);
}
Py_XDECREF(f->f_back);
Py_DECREF(f->f_builtins);
Py_DECREF(f->f_globals);
Py_CLEAR(f->f_locals);
Py_CLEAR(f->f_trace);
Py_CLEAR(f->f_exc_type);
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
co = f->f_code;
if (co->co_zombieframe == NULL)
co->co_zombieframe = f;
else if (numfree < PyFrame_MAXFREELIST) {
++numfree;
f->f_back = free_list;
free_list = f;
}
else
PyObject_GC_Del(f);
Py_DECREF(co);
Py_TRASHCAN_SAFE_END(f)
}
static int
frame_traverse(PyFrameObject *f, visitproc visit, void *arg)
{
PyObject **fastlocals, **p;
int i, slots;
Py_VISIT(f->f_back);
Py_VISIT(f->f_code);
Py_VISIT(f->f_builtins);
Py_VISIT(f->f_globals);
Py_VISIT(f->f_locals);
Py_VISIT(f->f_trace);
Py_VISIT(f->f_exc_type);
Py_VISIT(f->f_exc_value);
Py_VISIT(f->f_exc_traceback);
/* locals */
slots = f->f_code->co_nlocals + PyTuple_GET_SIZE(f->f_code->co_cellvars) + PyTuple_GET_SIZE(f->f_code->co_freevars);
fastlocals = f->f_localsplus;
for (i = slots; --i >= 0; ++fastlocals)
Py_VISIT(*fastlocals);
/* stack */
if (f->f_stacktop != NULL) {
for (p = f->f_valuestack; p < f->f_stacktop; p++)
Py_VISIT(*p);
}
return 0;
}
static void
frame_clear(PyFrameObject *f)
{
PyObject **fastlocals, **p, **oldtop;
int i, slots;
/* Before anything else, make sure that this frame is clearly marked
* as being defunct! Else, e.g., a generator reachable from this
* frame may also point to this frame, believe itself to still be
* active, and try cleaning up this frame again.
*/
oldtop = f->f_stacktop;
f->f_stacktop = NULL;
Py_CLEAR(f->f_exc_type);
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
Py_CLEAR(f->f_trace);
/* locals */
slots = f->f_code->co_nlocals + PyTuple_GET_SIZE(f->f_code->co_cellvars) + PyTuple_GET_SIZE(f->f_code->co_freevars);
fastlocals = f->f_localsplus;
for (i = slots; --i >= 0; ++fastlocals)
Py_CLEAR(*fastlocals);
/* stack */
if (oldtop != NULL) {
for (p = f->f_valuestack; p < oldtop; p++)
Py_CLEAR(*p);
}
}
static PyObject *
frame_sizeof(PyFrameObject *f)
{
Py_ssize_t res, extras, ncells, nfrees;
ncells = PyTuple_GET_SIZE(f->f_code->co_cellvars);
nfrees = PyTuple_GET_SIZE(f->f_code->co_freevars);
extras = f->f_code->co_stacksize + f->f_code->co_nlocals +
ncells + nfrees;
/* subtract one as it is already included in PyFrameObject */
res = sizeof(PyFrameObject) + (extras-1) * sizeof(PyObject *);
return PyInt_FromSsize_t(res);
}
PyDoc_STRVAR(sizeof__doc__,
"F.__sizeof__() -> size of F in memory, in bytes");
static PyMethodDef frame_methods[] = {
{"__sizeof__", (PyCFunction)frame_sizeof, METH_NOARGS,
sizeof__doc__},
{NULL, NULL} /* sentinel */
};
PyTypeObject PyFrame_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"frame",
sizeof(PyFrameObject),
sizeof(PyObject *),
(destructor)frame_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
PyObject_GenericSetAttr, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
0, /* tp_doc */
(traverseproc)frame_traverse, /* tp_traverse */
(inquiry)frame_clear, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
frame_methods, /* tp_methods */
frame_memberlist, /* tp_members */
frame_getsetlist, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
};
static PyObject *builtin_object;
int _PyFrame_Init()
{
builtin_object = PyString_InternFromString("__builtins__");
if (builtin_object == NULL)
return 0;
return 1;
}
PyFrameObject *
PyFrame_New(PyThreadState *tstate, PyCodeObject *code, PyObject *globals,
PyObject *locals)
{
PyFrameObject *back = tstate->frame;
PyFrameObject *f;
PyObject *builtins;
Py_ssize_t i;
#ifdef Py_DEBUG
if (code == NULL || globals == NULL || !PyDict_Check(globals) ||
(locals != NULL && !PyMapping_Check(locals))) {
PyErr_BadInternalCall();
return NULL;
}
#endif
if (back == NULL || back->f_globals != globals) {
builtins = PyDict_GetItem(globals, builtin_object);
if (builtins) {
if (PyModule_Check(builtins)) {
builtins = PyModule_GetDict(builtins);
assert(!builtins || PyDict_Check(builtins));
}
else if (!PyDict_Check(builtins))
builtins = NULL;
}
if (builtins == NULL) {
/* No builtins! Make up a minimal one
Give them 'None', at least. */
builtins = PyDict_New();
if (builtins == NULL ||
PyDict_SetItemString(
builtins, "None", Py_None) < 0)
return NULL;
}
else
Py_INCREF(builtins);
}
else {
/* If we share the globals, we share the builtins.
Save a lookup and a call. */
builtins = back->f_builtins;
assert(builtins != NULL && PyDict_Check(builtins));
Py_INCREF(builtins);
}
if (code->co_zombieframe != NULL) {
f = code->co_zombieframe;
code->co_zombieframe = NULL;
_Py_NewReference((PyObject *)f);
assert(f->f_code == code);
}
else {
Py_ssize_t extras, ncells, nfrees;
ncells = PyTuple_GET_SIZE(code->co_cellvars);
nfrees = PyTuple_GET_SIZE(code->co_freevars);
extras = code->co_stacksize + code->co_nlocals + ncells +
nfrees;
if (free_list == NULL) {
f = PyObject_GC_NewVar(PyFrameObject, &PyFrame_Type,
extras);
if (f == NULL) {
Py_DECREF(builtins);
return NULL;
}
}
else {
assert(numfree > 0);
--numfree;
f = free_list;
free_list = free_list->f_back;
if (Py_SIZE(f) < extras) {
f = PyObject_GC_Resize(PyFrameObject, f, extras);
if (f == NULL) {
Py_DECREF(builtins);
return NULL;
}
}
_Py_NewReference((PyObject *)f);
}
f->f_code = code;
extras = code->co_nlocals + ncells + nfrees;
f->f_valuestack = f->f_localsplus + extras;
for (i=0; i<extras; i++)
f->f_localsplus[i] = NULL;
f->f_locals = NULL;
f->f_trace = NULL;
f->f_exc_type = f->f_exc_value = f->f_exc_traceback = NULL;
}
f->f_stacktop = f->f_valuestack;
f->f_builtins = builtins;
Py_XINCREF(back);
f->f_back = back;
Py_INCREF(code);
Py_INCREF(globals);
f->f_globals = globals;
/* Most functions have CO_NEWLOCALS and CO_OPTIMIZED set. */
if ((code->co_flags & (CO_NEWLOCALS | CO_OPTIMIZED)) ==
(CO_NEWLOCALS | CO_OPTIMIZED))
; /* f_locals = NULL; will be set by PyFrame_FastToLocals() */
else if (code->co_flags & CO_NEWLOCALS) {
locals = PyDict_New();
if (locals == NULL) {
Py_DECREF(f);
return NULL;
}
f->f_locals = locals;
}
else {
if (locals == NULL)
locals = globals;
Py_INCREF(locals);
f->f_locals = locals;
}
f->f_tstate = tstate;
f->f_lasti = -1;
f->f_lineno = code->co_firstlineno;
f->f_iblock = 0;
_PyObject_GC_TRACK(f);
return f;
}
/* Block management */
void
PyFrame_BlockSetup(PyFrameObject *f, int type, int handler, int level)
{
PyTryBlock *b;
if (f->f_iblock >= CO_MAXBLOCKS)
Py_FatalError("XXX block stack overflow");
b = &f->f_blockstack[f->f_iblock++];
b->b_type = type;
b->b_level = level;
b->b_handler = handler;
}
PyTryBlock *
PyFrame_BlockPop(PyFrameObject *f)
{
PyTryBlock *b;
if (f->f_iblock <= 0)
Py_FatalError("XXX block stack underflow");
b = &f->f_blockstack[--f->f_iblock];
return b;
}
/* Convert between "fast" version of locals and dictionary version.
map and values are input arguments. map is a tuple of strings.
values is an array of PyObject*. At index i, map[i] is the name of
the variable with value values[i]. The function copies the first
nmap variable from map/values into dict. If values[i] is NULL,
the variable is deleted from dict.
If deref is true, then the values being copied are cell variables
and the value is extracted from the cell variable before being put
in dict.
Exceptions raised while modifying the dict are silently ignored,
because there is no good way to report them.
*/
static void
map_to_dict(PyObject *map, Py_ssize_t nmap, PyObject *dict, PyObject **values,
int deref)
{
Py_ssize_t j;
assert(PyTuple_Check(map));
assert(PyDict_Check(dict));
assert(PyTuple_Size(map) >= nmap);
for (j = nmap; --j >= 0; ) {
PyObject *key = PyTuple_GET_ITEM(map, j);
PyObject *value = values[j];
assert(PyString_Check(key));
if (deref) {
assert(PyCell_Check(value));
value = PyCell_GET(value);
}
if (value == NULL) {
if (PyObject_DelItem(dict, key) != 0)
PyErr_Clear();
}
else {
if (PyObject_SetItem(dict, key, value) != 0)
PyErr_Clear();
}
}
}
/* Copy values from the "locals" dict into the fast locals.
dict is an input argument containing string keys representing
variables names and arbitrary PyObject* as values.
map and values are input arguments. map is a tuple of strings.
values is an array of PyObject*. At index i, map[i] is the name of
the variable with value values[i]. The function copies the first
nmap variable from map/values into dict. If values[i] is NULL,
the variable is deleted from dict.
If deref is true, then the values being copied are cell variables
and the value is extracted from the cell variable before being put
in dict. If clear is true, then variables in map but not in dict
are set to NULL in map; if clear is false, variables missing in
dict are ignored.
Exceptions raised while modifying the dict are silently ignored,
because there is no good way to report them.
*/
static void
dict_to_map(PyObject *map, Py_ssize_t nmap, PyObject *dict, PyObject **values,
int deref, int clear)
{
Py_ssize_t j;
assert(PyTuple_Check(map));
assert(PyDict_Check(dict));
assert(PyTuple_Size(map) >= nmap);
for (j = nmap; --j >= 0; ) {
PyObject *key = PyTuple_GET_ITEM(map, j);
PyObject *value = PyObject_GetItem(dict, key);
assert(PyString_Check(key));
/* We only care about NULLs if clear is true. */
if (value == NULL) {
PyErr_Clear();
if (!clear)
continue;
}
if (deref) {
assert(PyCell_Check(values[j]));
if (PyCell_GET(values[j]) != value) {
if (PyCell_Set(values[j], value) < 0)
PyErr_Clear();
}
} else if (values[j] != value) {
Py_XINCREF(value);
Py_XDECREF(values[j]);
values[j] = value;
}
Py_XDECREF(value);
}
}
void
PyFrame_FastToLocals(PyFrameObject *f)
{
/* Merge fast locals into f->f_locals */
PyObject *locals, *map;
PyObject **fast;
PyObject *error_type, *error_value, *error_traceback;
PyCodeObject *co;
Py_ssize_t j;
int ncells, nfreevars;
if (f == NULL)
return;
locals = f->f_locals;
if (locals == NULL) {
locals = f->f_locals = PyDict_New();
if (locals == NULL) {
PyErr_Clear(); /* Can't report it :-( */
return;
}
}
co = f->f_code;
map = co->co_varnames;
if (!PyTuple_Check(map))
return;
PyErr_Fetch(&error_type, &error_value, &error_traceback);
fast = f->f_localsplus;
j = PyTuple_GET_SIZE(map);
if (j > co->co_nlocals)
j = co->co_nlocals;
if (co->co_nlocals)
map_to_dict(map, j, locals, fast, 0);
ncells = PyTuple_GET_SIZE(co->co_cellvars);
nfreevars = PyTuple_GET_SIZE(co->co_freevars);
if (ncells || nfreevars) {
map_to_dict(co->co_cellvars, ncells,
locals, fast + co->co_nlocals, 1);
/* If the namespace is unoptimized, then one of the
following cases applies:
1. It does not contain free variables, because it
uses import * or is a top-level namespace.
2. It is a class namespace.
We don't want to accidentally copy free variables
into the locals dict used by the class.
*/
if (co->co_flags & CO_OPTIMIZED) {
map_to_dict(co->co_freevars, nfreevars,
locals, fast + co->co_nlocals + ncells, 1);
}
}
PyErr_Restore(error_type, error_value, error_traceback);
}
void
PyFrame_LocalsToFast(PyFrameObject *f, int clear)
{
/* Merge f->f_locals into fast locals */
PyObject *locals, *map;
PyObject **fast;
PyObject *error_type, *error_value, *error_traceback;
PyCodeObject *co;
Py_ssize_t j;
int ncells, nfreevars;
if (f == NULL)
return;
locals = f->f_locals;
co = f->f_code;
map = co->co_varnames;
if (locals == NULL)
return;
if (!PyTuple_Check(map))
return;
PyErr_Fetch(&error_type, &error_value, &error_traceback);
fast = f->f_localsplus;
j = PyTuple_GET_SIZE(map);
if (j > co->co_nlocals)
j = co->co_nlocals;
if (co->co_nlocals)
dict_to_map(co->co_varnames, j, locals, fast, 0, clear);
ncells = PyTuple_GET_SIZE(co->co_cellvars);
nfreevars = PyTuple_GET_SIZE(co->co_freevars);
if (ncells || nfreevars) {
dict_to_map(co->co_cellvars, ncells,
locals, fast + co->co_nlocals, 1, clear);
/* Same test as in PyFrame_FastToLocals() above. */
if (co->co_flags & CO_OPTIMIZED) {
dict_to_map(co->co_freevars, nfreevars,
locals, fast + co->co_nlocals + ncells, 1,
clear);
}
}
PyErr_Restore(error_type, error_value, error_traceback);
}
/* Clear out the free list */
int
PyFrame_ClearFreeList(void)
{
int freelist_size = numfree;
while (free_list != NULL) {
PyFrameObject *f = free_list;
free_list = free_list->f_back;
PyObject_GC_Del(f);
--numfree;
}
assert(numfree == 0);
return freelist_size;
}
void
PyFrame_Fini(void)
{
(void)PyFrame_ClearFreeList();
Py_XDECREF(builtin_object);
builtin_object = NULL;
}

View File

@ -0,0 +1,895 @@
/* Function object implementation */
#include "Python.h"
#include "code.h"
#include "eval.h"
#include "structmember.h"
PyObject *
PyFunction_New(PyObject *code, PyObject *globals)
{
PyFunctionObject *op = PyObject_GC_New(PyFunctionObject,
&PyFunction_Type);
static PyObject *__name__ = 0;
if (op != NULL) {
PyObject *doc;
PyObject *consts;
PyObject *module;
op->func_weakreflist = NULL;
Py_INCREF(code);
op->func_code = code;
Py_INCREF(globals);
op->func_globals = globals;
op->func_name = ((PyCodeObject *)code)->co_name;
Py_INCREF(op->func_name);
op->func_defaults = NULL; /* No default arguments */
op->func_closure = NULL;
consts = ((PyCodeObject *)code)->co_consts;
if (PyTuple_Size(consts) >= 1) {
doc = PyTuple_GetItem(consts, 0);
if (!PyString_Check(doc) && !PyUnicode_Check(doc))
doc = Py_None;
}
else
doc = Py_None;
Py_INCREF(doc);
op->func_doc = doc;
op->func_dict = NULL;
op->func_module = NULL;
/* __module__: If module name is in globals, use it.
Otherwise, use None.
*/
if (!__name__) {
__name__ = PyString_InternFromString("__name__");
if (!__name__) {
Py_DECREF(op);
return NULL;
}
}
module = PyDict_GetItem(globals, __name__);
if (module) {
Py_INCREF(module);
op->func_module = module;
}
}
else
return NULL;
_PyObject_GC_TRACK(op);
return (PyObject *)op;
}
PyObject *
PyFunction_GetCode(PyObject *op)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyFunctionObject *) op) -> func_code;
}
PyObject *
PyFunction_GetGlobals(PyObject *op)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyFunctionObject *) op) -> func_globals;
}
PyObject *
PyFunction_GetModule(PyObject *op)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyFunctionObject *) op) -> func_module;
}
PyObject *
PyFunction_GetDefaults(PyObject *op)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyFunctionObject *) op) -> func_defaults;
}
int
PyFunction_SetDefaults(PyObject *op, PyObject *defaults)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return -1;
}
if (defaults == Py_None)
defaults = NULL;
else if (defaults && PyTuple_Check(defaults)) {
Py_INCREF(defaults);
}
else {
PyErr_SetString(PyExc_SystemError, "non-tuple default args");
return -1;
}
Py_XDECREF(((PyFunctionObject *) op) -> func_defaults);
((PyFunctionObject *) op) -> func_defaults = defaults;
return 0;
}
PyObject *
PyFunction_GetClosure(PyObject *op)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyFunctionObject *) op) -> func_closure;
}
int
PyFunction_SetClosure(PyObject *op, PyObject *closure)
{
if (!PyFunction_Check(op)) {
PyErr_BadInternalCall();
return -1;
}
if (closure == Py_None)
closure = NULL;
else if (PyTuple_Check(closure)) {
Py_INCREF(closure);
}
else {
PyErr_Format(PyExc_SystemError,
"expected tuple for closure, got '%.100s'",
closure->ob_type->tp_name);
return -1;
}
Py_XDECREF(((PyFunctionObject *) op) -> func_closure);
((PyFunctionObject *) op) -> func_closure = closure;
return 0;
}
/* Methods */
#define OFF(x) offsetof(PyFunctionObject, x)
static PyMemberDef func_memberlist[] = {
{"func_closure", T_OBJECT, OFF(func_closure),
RESTRICTED|READONLY},
{"__closure__", T_OBJECT, OFF(func_closure),
RESTRICTED|READONLY},
{"func_doc", T_OBJECT, OFF(func_doc), PY_WRITE_RESTRICTED},
{"__doc__", T_OBJECT, OFF(func_doc), PY_WRITE_RESTRICTED},
{"func_globals", T_OBJECT, OFF(func_globals),
RESTRICTED|READONLY},
{"__globals__", T_OBJECT, OFF(func_globals),
RESTRICTED|READONLY},
{"__module__", T_OBJECT, OFF(func_module), PY_WRITE_RESTRICTED},
{NULL} /* Sentinel */
};
static int
restricted(void)
{
if (!PyEval_GetRestricted())
return 0;
PyErr_SetString(PyExc_RuntimeError,
"function attributes not accessible in restricted mode");
return 1;
}
static PyObject *
func_get_dict(PyFunctionObject *op)
{
if (restricted())
return NULL;
if (op->func_dict == NULL) {
op->func_dict = PyDict_New();
if (op->func_dict == NULL)
return NULL;
}
Py_INCREF(op->func_dict);
return op->func_dict;
}
static int
func_set_dict(PyFunctionObject *op, PyObject *value)
{
PyObject *tmp;
if (restricted())
return -1;
/* It is illegal to del f.func_dict */
if (value == NULL) {
PyErr_SetString(PyExc_TypeError,
"function's dictionary may not be deleted");
return -1;
}
/* Can only set func_dict to a dictionary */
if (!PyDict_Check(value)) {
PyErr_SetString(PyExc_TypeError,
"setting function's dictionary to a non-dict");
return -1;
}
tmp = op->func_dict;
Py_INCREF(value);
op->func_dict = value;
Py_XDECREF(tmp);
return 0;
}
static PyObject *
func_get_code(PyFunctionObject *op)
{
if (restricted())
return NULL;
Py_INCREF(op->func_code);
return op->func_code;
}
static int
func_set_code(PyFunctionObject *op, PyObject *value)
{
PyObject *tmp;
Py_ssize_t nfree, nclosure;
if (restricted())
return -1;
/* Not legal to del f.func_code or to set it to anything
* other than a code object. */
if (value == NULL || !PyCode_Check(value)) {
PyErr_SetString(PyExc_TypeError,
"__code__ must be set to a code object");
return -1;
}
nfree = PyCode_GetNumFree((PyCodeObject *)value);
nclosure = (op->func_closure == NULL ? 0 :
PyTuple_GET_SIZE(op->func_closure));
if (nclosure != nfree) {
PyErr_Format(PyExc_ValueError,
"%s() requires a code object with %zd free vars,"
" not %zd",
PyString_AsString(op->func_name),
nclosure, nfree);
return -1;
}
tmp = op->func_code;
Py_INCREF(value);
op->func_code = value;
Py_DECREF(tmp);
return 0;
}
static PyObject *
func_get_name(PyFunctionObject *op)
{
Py_INCREF(op->func_name);
return op->func_name;
}
static int
func_set_name(PyFunctionObject *op, PyObject *value)
{
PyObject *tmp;
if (restricted())
return -1;
/* Not legal to del f.func_name or to set it to anything
* other than a string object. */
if (value == NULL || !PyString_Check(value)) {
PyErr_SetString(PyExc_TypeError,
"__name__ must be set to a string object");
return -1;
}
tmp = op->func_name;
Py_INCREF(value);
op->func_name = value;
Py_DECREF(tmp);
return 0;
}
static PyObject *
func_get_defaults(PyFunctionObject *op)
{
if (restricted())
return NULL;
if (op->func_defaults == NULL) {
Py_INCREF(Py_None);
return Py_None;
}
Py_INCREF(op->func_defaults);
return op->func_defaults;
}
static int
func_set_defaults(PyFunctionObject *op, PyObject *value)
{
PyObject *tmp;
if (restricted())
return -1;
/* Legal to del f.func_defaults.
* Can only set func_defaults to NULL or a tuple. */
if (value == Py_None)
value = NULL;
if (value != NULL && !PyTuple_Check(value)) {
PyErr_SetString(PyExc_TypeError,
"__defaults__ must be set to a tuple object");
return -1;
}
tmp = op->func_defaults;
Py_XINCREF(value);
op->func_defaults = value;
Py_XDECREF(tmp);
return 0;
}
static PyGetSetDef func_getsetlist[] = {
{"func_code", (getter)func_get_code, (setter)func_set_code},
{"__code__", (getter)func_get_code, (setter)func_set_code},
{"func_defaults", (getter)func_get_defaults,
(setter)func_set_defaults},
{"__defaults__", (getter)func_get_defaults,
(setter)func_set_defaults},
{"func_dict", (getter)func_get_dict, (setter)func_set_dict},
{"__dict__", (getter)func_get_dict, (setter)func_set_dict},
{"func_name", (getter)func_get_name, (setter)func_set_name},
{"__name__", (getter)func_get_name, (setter)func_set_name},
{NULL} /* Sentinel */
};
PyDoc_STRVAR(func_doc,
"function(code, globals[, name[, argdefs[, closure]]])\n\
\n\
Create a function object from a code object and a dictionary.\n\
The optional name string overrides the name from the code object.\n\
The optional argdefs tuple specifies the default argument values.\n\
The optional closure tuple supplies the bindings for free variables.");
/* func_new() maintains the following invariants for closures. The
closure must correspond to the free variables of the code object.
if len(code.co_freevars) == 0:
closure = NULL
else:
len(closure) == len(code.co_freevars)
for every elt in closure, type(elt) == cell
*/
static PyObject *
func_new(PyTypeObject* type, PyObject* args, PyObject* kw)
{
PyCodeObject *code;
PyObject *globals;
PyObject *name = Py_None;
PyObject *defaults = Py_None;
PyObject *closure = Py_None;
PyFunctionObject *newfunc;
Py_ssize_t nfree, nclosure;
static char *kwlist[] = {"code", "globals", "name",
"argdefs", "closure", 0};
if (!PyArg_ParseTupleAndKeywords(args, kw, "O!O!|OOO:function",
kwlist,
&PyCode_Type, &code,
&PyDict_Type, &globals,
&name, &defaults, &closure))
return NULL;
if (name != Py_None && !PyString_Check(name)) {
PyErr_SetString(PyExc_TypeError,
"arg 3 (name) must be None or string");
return NULL;
}
if (defaults != Py_None && !PyTuple_Check(defaults)) {
PyErr_SetString(PyExc_TypeError,
"arg 4 (defaults) must be None or tuple");
return NULL;
}
nfree = PyTuple_GET_SIZE(code->co_freevars);
if (!PyTuple_Check(closure)) {
if (nfree && closure == Py_None) {
PyErr_SetString(PyExc_TypeError,
"arg 5 (closure) must be tuple");
return NULL;
}
else if (closure != Py_None) {
PyErr_SetString(PyExc_TypeError,
"arg 5 (closure) must be None or tuple");
return NULL;
}
}
/* check that the closure is well-formed */
nclosure = closure == Py_None ? 0 : PyTuple_GET_SIZE(closure);
if (nfree != nclosure)
return PyErr_Format(PyExc_ValueError,
"%s requires closure of length %zd, not %zd",
PyString_AS_STRING(code->co_name),
nfree, nclosure);
if (nclosure) {
Py_ssize_t i;
for (i = 0; i < nclosure; i++) {
PyObject *o = PyTuple_GET_ITEM(closure, i);
if (!PyCell_Check(o)) {
return PyErr_Format(PyExc_TypeError,
"arg 5 (closure) expected cell, found %s",
o->ob_type->tp_name);
}
}
}
newfunc = (PyFunctionObject *)PyFunction_New((PyObject *)code,
globals);
if (newfunc == NULL)
return NULL;
if (name != Py_None) {
Py_INCREF(name);
Py_DECREF(newfunc->func_name);
newfunc->func_name = name;
}
if (defaults != Py_None) {
Py_INCREF(defaults);
newfunc->func_defaults = defaults;
}
if (closure != Py_None) {
Py_INCREF(closure);
newfunc->func_closure = closure;
}
return (PyObject *)newfunc;
}
static void
func_dealloc(PyFunctionObject *op)
{
_PyObject_GC_UNTRACK(op);
if (op->func_weakreflist != NULL)
PyObject_ClearWeakRefs((PyObject *) op);
Py_DECREF(op->func_code);
Py_DECREF(op->func_globals);
Py_XDECREF(op->func_module);
Py_DECREF(op->func_name);
Py_XDECREF(op->func_defaults);
Py_XDECREF(op->func_doc);
Py_XDECREF(op->func_dict);
Py_XDECREF(op->func_closure);
PyObject_GC_Del(op);
}
static PyObject*
func_repr(PyFunctionObject *op)
{
return PyString_FromFormat("<function %s at %p>",
PyString_AsString(op->func_name),
op);
}
static int
func_traverse(PyFunctionObject *f, visitproc visit, void *arg)
{
Py_VISIT(f->func_code);
Py_VISIT(f->func_globals);
Py_VISIT(f->func_module);
Py_VISIT(f->func_defaults);
Py_VISIT(f->func_doc);
Py_VISIT(f->func_name);
Py_VISIT(f->func_dict);
Py_VISIT(f->func_closure);
return 0;
}
static PyObject *
function_call(PyObject *func, PyObject *arg, PyObject *kw)
{
PyObject *result;
PyObject *argdefs;
PyObject *kwtuple = NULL;
PyObject **d, **k;
Py_ssize_t nk, nd;
argdefs = PyFunction_GET_DEFAULTS(func);
if (argdefs != NULL && PyTuple_Check(argdefs)) {
d = &PyTuple_GET_ITEM((PyTupleObject *)argdefs, 0);
nd = PyTuple_GET_SIZE(argdefs);
}
else {
d = NULL;
nd = 0;
}
if (kw != NULL && PyDict_Check(kw)) {
Py_ssize_t pos, i;
nk = PyDict_Size(kw);
kwtuple = PyTuple_New(2*nk);
if (kwtuple == NULL)
return NULL;
k = &PyTuple_GET_ITEM(kwtuple, 0);
pos = i = 0;
while (PyDict_Next(kw, &pos, &k[i], &k[i+1])) {
Py_INCREF(k[i]);
Py_INCREF(k[i+1]);
i += 2;
}
nk = i/2;
}
else {
k = NULL;
nk = 0;
}
result = PyEval_EvalCodeEx(
(PyCodeObject *)PyFunction_GET_CODE(func),
PyFunction_GET_GLOBALS(func), (PyObject *)NULL,
&PyTuple_GET_ITEM(arg, 0), PyTuple_GET_SIZE(arg),
k, nk, d, nd,
PyFunction_GET_CLOSURE(func));
Py_XDECREF(kwtuple);
return result;
}
/* Bind a function to an object */
static PyObject *
func_descr_get(PyObject *func, PyObject *obj, PyObject *type)
{
if (obj == Py_None)
obj = NULL;
return PyMethod_New(func, obj, type);
}
PyTypeObject PyFunction_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"function",
sizeof(PyFunctionObject),
0,
(destructor)func_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)func_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
function_call, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
PyObject_GenericSetAttr, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
func_doc, /* tp_doc */
(traverseproc)func_traverse, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
offsetof(PyFunctionObject, func_weakreflist), /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
func_memberlist, /* tp_members */
func_getsetlist, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
func_descr_get, /* tp_descr_get */
0, /* tp_descr_set */
offsetof(PyFunctionObject, func_dict), /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
func_new, /* tp_new */
};
/* Class method object */
/* A class method receives the class as implicit first argument,
just like an instance method receives the instance.
To declare a class method, use this idiom:
class C:
def f(cls, arg1, arg2, ...): ...
f = classmethod(f)
It can be called either on the class (e.g. C.f()) or on an instance
(e.g. C().f()); the instance is ignored except for its class.
If a class method is called for a derived class, the derived class
object is passed as the implied first argument.
Class methods are different than C++ or Java static methods.
If you want those, see static methods below.
*/
typedef struct {
PyObject_HEAD
PyObject *cm_callable;
} classmethod;
static void
cm_dealloc(classmethod *cm)
{
_PyObject_GC_UNTRACK((PyObject *)cm);
Py_XDECREF(cm->cm_callable);
Py_TYPE(cm)->tp_free((PyObject *)cm);
}
static int
cm_traverse(classmethod *cm, visitproc visit, void *arg)
{
Py_VISIT(cm->cm_callable);
return 0;
}
static int
cm_clear(classmethod *cm)
{
Py_CLEAR(cm->cm_callable);
return 0;
}
static PyObject *
cm_descr_get(PyObject *self, PyObject *obj, PyObject *type)
{
classmethod *cm = (classmethod *)self;
if (cm->cm_callable == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"uninitialized classmethod object");
return NULL;
}
if (type == NULL)
type = (PyObject *)(Py_TYPE(obj));
return PyMethod_New(cm->cm_callable,
type, (PyObject *)(Py_TYPE(type)));
}
static int
cm_init(PyObject *self, PyObject *args, PyObject *kwds)
{
classmethod *cm = (classmethod *)self;
PyObject *callable;
if (!PyArg_UnpackTuple(args, "classmethod", 1, 1, &callable))
return -1;
if (!_PyArg_NoKeywords("classmethod", kwds))
return -1;
Py_INCREF(callable);
cm->cm_callable = callable;
return 0;
}
static PyMemberDef cm_memberlist[] = {
{"__func__", T_OBJECT, offsetof(classmethod, cm_callable), READONLY},
{NULL} /* Sentinel */
};
PyDoc_STRVAR(classmethod_doc,
"classmethod(function) -> method\n\
\n\
Convert a function to be a class method.\n\
\n\
A class method receives the class as implicit first argument,\n\
just like an instance method receives the instance.\n\
To declare a class method, use this idiom:\n\
\n\
class C:\n\
def f(cls, arg1, arg2, ...): ...\n\
f = classmethod(f)\n\
\n\
It can be called either on the class (e.g. C.f()) or on an instance\n\
(e.g. C().f()). The instance is ignored except for its class.\n\
If a class method is called for a derived class, the derived class\n\
object is passed as the implied first argument.\n\
\n\
Class methods are different than C++ or Java static methods.\n\
If you want those, see the staticmethod builtin.");
PyTypeObject PyClassMethod_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"classmethod",
sizeof(classmethod),
0,
(destructor)cm_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,
classmethod_doc, /* tp_doc */
(traverseproc)cm_traverse, /* tp_traverse */
(inquiry)cm_clear, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
cm_memberlist, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
cm_descr_get, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
cm_init, /* tp_init */
PyType_GenericAlloc, /* tp_alloc */
PyType_GenericNew, /* tp_new */
PyObject_GC_Del, /* tp_free */
};
PyObject *
PyClassMethod_New(PyObject *callable)
{
classmethod *cm = (classmethod *)
PyType_GenericAlloc(&PyClassMethod_Type, 0);
if (cm != NULL) {
Py_INCREF(callable);
cm->cm_callable = callable;
}
return (PyObject *)cm;
}
/* Static method object */
/* A static method does not receive an implicit first argument.
To declare a static method, use this idiom:
class C:
def f(arg1, arg2, ...): ...
f = staticmethod(f)
It can be called either on the class (e.g. C.f()) or on an instance
(e.g. C().f()); the instance is ignored except for its class.
Static methods in Python are similar to those found in Java or C++.
For a more advanced concept, see class methods above.
*/
typedef struct {
PyObject_HEAD
PyObject *sm_callable;
} staticmethod;
static void
sm_dealloc(staticmethod *sm)
{
_PyObject_GC_UNTRACK((PyObject *)sm);
Py_XDECREF(sm->sm_callable);
Py_TYPE(sm)->tp_free((PyObject *)sm);
}
static int
sm_traverse(staticmethod *sm, visitproc visit, void *arg)
{
Py_VISIT(sm->sm_callable);
return 0;
}
static int
sm_clear(staticmethod *sm)
{
Py_CLEAR(sm->sm_callable);
return 0;
}
static PyObject *
sm_descr_get(PyObject *self, PyObject *obj, PyObject *type)
{
staticmethod *sm = (staticmethod *)self;
if (sm->sm_callable == NULL) {
PyErr_SetString(PyExc_RuntimeError,
"uninitialized staticmethod object");
return NULL;
}
Py_INCREF(sm->sm_callable);
return sm->sm_callable;
}
static int
sm_init(PyObject *self, PyObject *args, PyObject *kwds)
{
staticmethod *sm = (staticmethod *)self;
PyObject *callable;
if (!PyArg_UnpackTuple(args, "staticmethod", 1, 1, &callable))
return -1;
if (!_PyArg_NoKeywords("staticmethod", kwds))
return -1;
Py_INCREF(callable);
sm->sm_callable = callable;
return 0;
}
static PyMemberDef sm_memberlist[] = {
{"__func__", T_OBJECT, offsetof(staticmethod, sm_callable), READONLY},
{NULL} /* Sentinel */
};
PyDoc_STRVAR(staticmethod_doc,
"staticmethod(function) -> method\n\
\n\
Convert a function to be a static method.\n\
\n\
A static method does not receive an implicit first argument.\n\
To declare a static method, use this idiom:\n\
\n\
class C:\n\
def f(arg1, arg2, ...): ...\n\
f = staticmethod(f)\n\
\n\
It can be called either on the class (e.g. C.f()) or on an instance\n\
(e.g. C().f()). The instance is ignored except for its class.\n\
\n\
Static methods in Python are similar to those found in Java or C++.\n\
For a more advanced concept, see the classmethod builtin.");
PyTypeObject PyStaticMethod_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"staticmethod",
sizeof(staticmethod),
0,
(destructor)sm_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,
staticmethod_doc, /* tp_doc */
(traverseproc)sm_traverse, /* tp_traverse */
(inquiry)sm_clear, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
sm_memberlist, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
sm_descr_get, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
sm_init, /* tp_init */
PyType_GenericAlloc, /* tp_alloc */
PyType_GenericNew, /* tp_new */
PyObject_GC_Del, /* tp_free */
};
PyObject *
PyStaticMethod_New(PyObject *callable)
{
staticmethod *sm = (staticmethod *)
PyType_GenericAlloc(&PyStaticMethod_Type, 0);
if (sm != NULL) {
Py_INCREF(callable);
sm->sm_callable = callable;
}
return (PyObject *)sm;
}

View File

@ -0,0 +1,417 @@
/* Generator object implementation */
#include "Python.h"
#include "frameobject.h"
#include "genobject.h"
#include "ceval.h"
#include "structmember.h"
#include "opcode.h"
static int
gen_traverse(PyGenObject *gen, visitproc visit, void *arg)
{
Py_VISIT((PyObject *)gen->gi_frame);
Py_VISIT(gen->gi_code);
return 0;
}
static void
gen_dealloc(PyGenObject *gen)
{
PyObject *self = (PyObject *) gen;
_PyObject_GC_UNTRACK(gen);
if (gen->gi_weakreflist != NULL)
PyObject_ClearWeakRefs(self);
_PyObject_GC_TRACK(self);
if (gen->gi_frame != NULL && gen->gi_frame->f_stacktop != NULL) {
/* Generator is paused, so we need to close */
Py_TYPE(gen)->tp_del(self);
if (self->ob_refcnt > 0)
return; /* resurrected. :( */
}
_PyObject_GC_UNTRACK(self);
Py_CLEAR(gen->gi_frame);
Py_CLEAR(gen->gi_code);
PyObject_GC_Del(gen);
}
static PyObject *
gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
{
PyThreadState *tstate = PyThreadState_GET();
PyFrameObject *f = gen->gi_frame;
PyObject *result;
if (gen->gi_running) {
PyErr_SetString(PyExc_ValueError,
"generator already executing");
return NULL;
}
if (f==NULL || f->f_stacktop == NULL) {
/* Only set exception if called from send() */
if (arg && !exc)
PyErr_SetNone(PyExc_StopIteration);
return NULL;
}
if (f->f_lasti == -1) {
if (arg && arg != Py_None) {
PyErr_SetString(PyExc_TypeError,
"can't send non-None value to a "
"just-started generator");
return NULL;
}
} else {
/* Push arg onto the frame's value stack */
result = arg ? arg : Py_None;
Py_INCREF(result);
*(f->f_stacktop++) = result;
}
/* Generators always return to their most recent caller, not
* necessarily their creator. */
f->f_tstate = tstate;
Py_XINCREF(tstate->frame);
assert(f->f_back == NULL);
f->f_back = tstate->frame;
gen->gi_running = 1;
result = PyEval_EvalFrameEx(f, exc);
gen->gi_running = 0;
/* Don't keep the reference to f_back any longer than necessary. It
* may keep a chain of frames alive or it could create a reference
* cycle. */
assert(f->f_back == tstate->frame);
Py_CLEAR(f->f_back);
/* Clear the borrowed reference to the thread state */
f->f_tstate = NULL;
/* If the generator just returned (as opposed to yielding), signal
* that the generator is exhausted. */
if (result == Py_None && f->f_stacktop == NULL) {
Py_DECREF(result);
result = NULL;
/* Set exception if not called by gen_iternext() */
if (arg)
PyErr_SetNone(PyExc_StopIteration);
}
if (!result || f->f_stacktop == NULL) {
/* generator can't be rerun, so release the frame */
Py_DECREF(f);
gen->gi_frame = NULL;
}
return result;
}
PyDoc_STRVAR(send_doc,
"send(arg) -> send 'arg' into generator,\n\
return next yielded value or raise StopIteration.");
static PyObject *
gen_send(PyGenObject *gen, PyObject *arg)
{
return gen_send_ex(gen, arg, 0);
}
PyDoc_STRVAR(close_doc,
"close() -> raise GeneratorExit inside generator.");
static PyObject *
gen_close(PyGenObject *gen, PyObject *args)
{
PyObject *retval;
PyErr_SetNone(PyExc_GeneratorExit);
retval = gen_send_ex(gen, Py_None, 1);
if (retval) {
Py_DECREF(retval);
PyErr_SetString(PyExc_RuntimeError,
"generator ignored GeneratorExit");
return NULL;
}
if (PyErr_ExceptionMatches(PyExc_StopIteration)
|| PyErr_ExceptionMatches(PyExc_GeneratorExit))
{
PyErr_Clear(); /* ignore these errors */
Py_INCREF(Py_None);
return Py_None;
}
return NULL;
}
static void
gen_del(PyObject *self)
{
PyObject *res;
PyObject *error_type, *error_value, *error_traceback;
PyGenObject *gen = (PyGenObject *)self;
if (gen->gi_frame == NULL || gen->gi_frame->f_stacktop == NULL)
/* Generator isn't paused, so no need to close */
return;
/* Temporarily resurrect the object. */
assert(self->ob_refcnt == 0);
self->ob_refcnt = 1;
/* Save the current exception, if any. */
PyErr_Fetch(&error_type, &error_value, &error_traceback);
res = gen_close(gen, NULL);
if (res == NULL)
PyErr_WriteUnraisable(self);
else
Py_DECREF(res);
/* Restore the saved exception. */
PyErr_Restore(error_type, error_value, error_traceback);
/* Undo the temporary resurrection; can't use DECREF here, it would
* cause a recursive call.
*/
assert(self->ob_refcnt > 0);
if (--self->ob_refcnt == 0)
return; /* this is the normal path out */
/* close() resurrected it! Make it look like the original Py_DECREF
* never happened.
*/
{
Py_ssize_t refcnt = self->ob_refcnt;
_Py_NewReference(self);
self->ob_refcnt = refcnt;
}
assert(PyType_IS_GC(self->ob_type) &&
_Py_AS_GC(self)->gc.gc_refs != _PyGC_REFS_UNTRACKED);
/* If Py_REF_DEBUG, _Py_NewReference bumped _Py_RefTotal, so
* we need to undo that. */
_Py_DEC_REFTOTAL;
/* If Py_TRACE_REFS, _Py_NewReference re-added self to the object
* chain, so no more to do there.
* If COUNT_ALLOCS, the original decref bumped tp_frees, and
* _Py_NewReference bumped tp_allocs: both of those need to be
* undone.
*/
#ifdef COUNT_ALLOCS
--self->ob_type->tp_frees;
--self->ob_type->tp_allocs;
#endif
}
PyDoc_STRVAR(throw_doc,
"throw(typ[,val[,tb]]) -> raise exception in generator,\n\
return next yielded value or raise StopIteration.");
static PyObject *
gen_throw(PyGenObject *gen, PyObject *args)
{
PyObject *typ;
PyObject *tb = NULL;
PyObject *val = NULL;
if (!PyArg_UnpackTuple(args, "throw", 1, 3, &typ, &val, &tb))
return NULL;
/* First, check the traceback argument, replacing None with
NULL. */
if (tb == Py_None)
tb = NULL;
else if (tb != NULL && !PyTraceBack_Check(tb)) {
PyErr_SetString(PyExc_TypeError,
"throw() third argument must be a traceback object");
return NULL;
}
Py_INCREF(typ);
Py_XINCREF(val);
Py_XINCREF(tb);
if (PyExceptionClass_Check(typ)) {
PyErr_NormalizeException(&typ, &val, &tb);
}
else if (PyExceptionInstance_Check(typ)) {
/* Raising an instance. The value should be a dummy. */
if (val && val != Py_None) {
PyErr_SetString(PyExc_TypeError,
"instance exception may not have a separate value");
goto failed_throw;
}
else {
/* Normalize to raise <class>, <instance> */
Py_XDECREF(val);
val = typ;
typ = PyExceptionInstance_Class(typ);
Py_INCREF(typ);
}
}
else {
/* Not something you can raise. throw() fails. */
PyErr_Format(PyExc_TypeError,
"exceptions must be classes, or instances, not %s",
typ->ob_type->tp_name);
goto failed_throw;
}
PyErr_Restore(typ, val, tb);
return gen_send_ex(gen, Py_None, 1);
failed_throw:
/* Didn't use our arguments, so restore their original refcounts */
Py_DECREF(typ);
Py_XDECREF(val);
Py_XDECREF(tb);
return NULL;
}
static PyObject *
gen_iternext(PyGenObject *gen)
{
return gen_send_ex(gen, NULL, 0);
}
static PyObject *
gen_repr(PyGenObject *gen)
{
char *code_name;
code_name = PyString_AsString(((PyCodeObject *)gen->gi_code)->co_name);
if (code_name == NULL)
return NULL;
return PyString_FromFormat("<generator object %.200s at %p>",
code_name, gen);
}
static PyObject *
gen_get_name(PyGenObject *gen)
{
PyObject *name = ((PyCodeObject *)gen->gi_code)->co_name;
Py_INCREF(name);
return name;
}
PyDoc_STRVAR(gen__name__doc__,
"Return the name of the generator's associated code object.");
static PyGetSetDef gen_getsetlist[] = {
{"__name__", (getter)gen_get_name, NULL, gen__name__doc__},
{NULL}
};
static PyMemberDef gen_memberlist[] = {
{"gi_frame", T_OBJECT, offsetof(PyGenObject, gi_frame), RO},
{"gi_running", T_INT, offsetof(PyGenObject, gi_running), RO},
{"gi_code", T_OBJECT, offsetof(PyGenObject, gi_code), RO},
{NULL} /* Sentinel */
};
static PyMethodDef gen_methods[] = {
{"send",(PyCFunction)gen_send, METH_O, send_doc},
{"throw",(PyCFunction)gen_throw, METH_VARARGS, throw_doc},
{"close",(PyCFunction)gen_close, METH_NOARGS, close_doc},
{NULL, NULL} /* Sentinel */
};
PyTypeObject PyGen_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"generator", /* tp_name */
sizeof(PyGenObject), /* tp_basicsize */
0, /* tp_itemsize */
/* methods */
(destructor)gen_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)gen_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
0, /* tp_doc */
(traverseproc)gen_traverse, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
offsetof(PyGenObject, gi_weakreflist), /* tp_weaklistoffset */
PyObject_SelfIter, /* tp_iter */
(iternextfunc)gen_iternext, /* tp_iternext */
gen_methods, /* tp_methods */
gen_memberlist, /* tp_members */
gen_getsetlist, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
0, /* tp_new */
0, /* tp_free */
0, /* tp_is_gc */
0, /* tp_bases */
0, /* tp_mro */
0, /* tp_cache */
0, /* tp_subclasses */
0, /* tp_weaklist */
gen_del, /* tp_del */
};
PyObject *
PyGen_New(PyFrameObject *f)
{
PyGenObject *gen = PyObject_GC_New(PyGenObject, &PyGen_Type);
if (gen == NULL) {
Py_DECREF(f);
return NULL;
}
gen->gi_frame = f;
Py_INCREF(f->f_code);
gen->gi_code = (PyObject *)(f->f_code);
gen->gi_running = 0;
gen->gi_weakreflist = NULL;
_PyObject_GC_TRACK(gen);
return (PyObject *)gen;
}
int
PyGen_NeedsFinalizing(PyGenObject *gen)
{
int i;
PyFrameObject *f = gen->gi_frame;
if (f == NULL || f->f_stacktop == NULL || f->f_iblock <= 0)
return 0; /* no frame or empty blockstack == no finalization */
/* Any block type besides a loop requires cleanup. */
i = f->f_iblock;
while (--i >= 0) {
if (f->f_blockstack[i].b_type != SETUP_LOOP)
return 1;
}
/* No blocks except loops, it's safe to skip finalization. */
return 0;
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,230 @@
/* Iterator objects */
#include "Python.h"
typedef struct {
PyObject_HEAD
long it_index;
PyObject *it_seq; /* Set to NULL when iterator is exhausted */
} seqiterobject;
PyObject *
PySeqIter_New(PyObject *seq)
{
seqiterobject *it;
if (!PySequence_Check(seq)) {
PyErr_BadInternalCall();
return NULL;
}
it = PyObject_GC_New(seqiterobject, &PySeqIter_Type);
if (it == NULL)
return NULL;
it->it_index = 0;
Py_INCREF(seq);
it->it_seq = seq;
_PyObject_GC_TRACK(it);
return (PyObject *)it;
}
static void
iter_dealloc(seqiterobject *it)
{
_PyObject_GC_UNTRACK(it);
Py_XDECREF(it->it_seq);
PyObject_GC_Del(it);
}
static int
iter_traverse(seqiterobject *it, visitproc visit, void *arg)
{
Py_VISIT(it->it_seq);
return 0;
}
static PyObject *
iter_iternext(PyObject *iterator)
{
seqiterobject *it;
PyObject *seq;
PyObject *result;
assert(PySeqIter_Check(iterator));
it = (seqiterobject *)iterator;
seq = it->it_seq;
if (seq == NULL)
return NULL;
result = PySequence_GetItem(seq, it->it_index);
if (result != NULL) {
it->it_index++;
return result;
}
if (PyErr_ExceptionMatches(PyExc_IndexError) ||
PyErr_ExceptionMatches(PyExc_StopIteration))
{
PyErr_Clear();
Py_DECREF(seq);
it->it_seq = NULL;
}
return NULL;
}
static PyObject *
iter_len(seqiterobject *it)
{
Py_ssize_t seqsize, len;
if (it->it_seq) {
seqsize = PySequence_Size(it->it_seq);
if (seqsize == -1)
return NULL;
len = seqsize - it->it_index;
if (len >= 0)
return PyInt_FromSsize_t(len);
}
return PyInt_FromLong(0);
}
PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list(it)).");
static PyMethodDef seqiter_methods[] = {
{"__length_hint__", (PyCFunction)iter_len, METH_NOARGS, length_hint_doc},
{NULL, NULL} /* sentinel */
};
PyTypeObject PySeqIter_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"iterator", /* tp_name */
sizeof(seqiterobject), /* tp_basicsize */
0, /* tp_itemsize */
/* methods */
(destructor)iter_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
0, /* tp_doc */
(traverseproc)iter_traverse, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
PyObject_SelfIter, /* tp_iter */
iter_iternext, /* tp_iternext */
seqiter_methods, /* tp_methods */
0, /* tp_members */
};
/* -------------------------------------- */
typedef struct {
PyObject_HEAD
PyObject *it_callable; /* Set to NULL when iterator is exhausted */
PyObject *it_sentinel; /* Set to NULL when iterator is exhausted */
} calliterobject;
PyObject *
PyCallIter_New(PyObject *callable, PyObject *sentinel)
{
calliterobject *it;
it = PyObject_GC_New(calliterobject, &PyCallIter_Type);
if (it == NULL)
return NULL;
Py_INCREF(callable);
it->it_callable = callable;
Py_INCREF(sentinel);
it->it_sentinel = sentinel;
_PyObject_GC_TRACK(it);
return (PyObject *)it;
}
static void
calliter_dealloc(calliterobject *it)
{
_PyObject_GC_UNTRACK(it);
Py_XDECREF(it->it_callable);
Py_XDECREF(it->it_sentinel);
PyObject_GC_Del(it);
}
static int
calliter_traverse(calliterobject *it, visitproc visit, void *arg)
{
Py_VISIT(it->it_callable);
Py_VISIT(it->it_sentinel);
return 0;
}
static PyObject *
calliter_iternext(calliterobject *it)
{
if (it->it_callable != NULL) {
PyObject *args = PyTuple_New(0);
PyObject *result;
if (args == NULL)
return NULL;
result = PyObject_Call(it->it_callable, args, NULL);
Py_DECREF(args);
if (result != NULL) {
int ok;
ok = PyObject_RichCompareBool(result,
it->it_sentinel,
Py_EQ);
if (ok == 0)
return result; /* Common case, fast path */
Py_DECREF(result);
if (ok > 0) {
Py_CLEAR(it->it_callable);
Py_CLEAR(it->it_sentinel);
}
}
else if (PyErr_ExceptionMatches(PyExc_StopIteration)) {
PyErr_Clear();
Py_CLEAR(it->it_callable);
Py_CLEAR(it->it_sentinel);
}
}
return NULL;
}
PyTypeObject PyCallIter_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"callable-iterator", /* tp_name */
sizeof(calliterobject), /* tp_basicsize */
0, /* tp_itemsize */
/* methods */
(destructor)calliter_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
0, /* tp_doc */
(traverseproc)calliter_traverse, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
PyObject_SelfIter, /* tp_iter */
(iternextfunc)calliter_iternext, /* tp_iternext */
0, /* tp_methods */
};

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,755 @@
Intro
-----
This describes an adaptive, stable, natural mergesort, modestly called
timsort (hey, I earned it <wink>). It has supernatural performance on many
kinds of partially ordered arrays (less than lg(N!) comparisons needed, and
as few as N-1), yet as fast as Python's previous highly tuned samplesort
hybrid on random arrays.
In a nutshell, the main routine marches over the array once, left to right,
alternately identifying the next run, then merging it into the previous
runs "intelligently". Everything else is complication for speed, and some
hard-won measure of memory efficiency.
Comparison with Python's Samplesort Hybrid
------------------------------------------
+ timsort can require a temp array containing as many as N//2 pointers,
which means as many as 2*N extra bytes on 32-bit boxes. It can be
expected to require a temp array this large when sorting random data; on
data with significant structure, it may get away without using any extra
heap memory. This appears to be the strongest argument against it, but
compared to the size of an object, 2 temp bytes worst-case (also expected-
case for random data) doesn't scare me much.
It turns out that Perl is moving to a stable mergesort, and the code for
that appears always to require a temp array with room for at least N
pointers. (Note that I wouldn't want to do that even if space weren't an
issue; I believe its efforts at memory frugality also save timsort
significant pointer-copying costs, and allow it to have a smaller working
set.)
+ Across about four hours of generating random arrays, and sorting them
under both methods, samplesort required about 1.5% more comparisons
(the program is at the end of this file).
+ In real life, this may be faster or slower on random arrays than
samplesort was, depending on platform quirks. Since it does fewer
comparisons on average, it can be expected to do better the more
expensive a comparison function is. OTOH, it does more data movement
(pointer copying) than samplesort, and that may negate its small
comparison advantage (depending on platform quirks) unless comparison
is very expensive.
+ On arrays with many kinds of pre-existing order, this blows samplesort out
of the water. It's significantly faster than samplesort even on some
cases samplesort was special-casing the snot out of. I believe that lists
very often do have exploitable partial order in real life, and this is the
strongest argument in favor of timsort (indeed, samplesort's special cases
for extreme partial order are appreciated by real users, and timsort goes
much deeper than those, in particular naturally covering every case where
someone has suggested "and it would be cool if list.sort() had a special
case for this too ... and for that ...").
+ Here are exact comparison counts across all the tests in sortperf.py,
when run with arguments "15 20 1".
Column Key:
*sort: random data
\sort: descending data
/sort: ascending data
3sort: ascending, then 3 random exchanges
+sort: ascending, then 10 random at the end
%sort: ascending, then randomly replace 1% of elements w/ random values
~sort: many duplicates
=sort: all equal
!sort: worst case scenario
First the trivial cases, trivial for samplesort because it special-cased
them, and trivial for timsort because it naturally works on runs. Within
an "n" block, the first line gives the # of compares done by samplesort,
the second line by timsort, and the third line is the percentage by
which the samplesort count exceeds the timsort count:
n \sort /sort =sort
------- ------ ------ ------
32768 32768 32767 32767 samplesort
32767 32767 32767 timsort
0.00% 0.00% 0.00% (samplesort - timsort) / timsort
65536 65536 65535 65535
65535 65535 65535
0.00% 0.00% 0.00%
131072 131072 131071 131071
131071 131071 131071
0.00% 0.00% 0.00%
262144 262144 262143 262143
262143 262143 262143
0.00% 0.00% 0.00%
524288 524288 524287 524287
524287 524287 524287
0.00% 0.00% 0.00%
1048576 1048576 1048575 1048575
1048575 1048575 1048575
0.00% 0.00% 0.00%
The algorithms are effectively identical in these cases, except that
timsort does one less compare in \sort.
Now for the more interesting cases. Where lg(x) is the logarithm of x to
the base 2 (e.g., lg(8)=3), lg(n!) is the information-theoretic limit for
the best any comparison-based sorting algorithm can do on average (across
all permutations). When a method gets significantly below that, it's
either astronomically lucky, or is finding exploitable structure in the
data.
n lg(n!) *sort 3sort +sort %sort ~sort !sort
------- ------- ------ ------- ------- ------ ------- --------
32768 444255 453096 453614 32908 452871 130491 469141 old
448885 33016 33007 50426 182083 65534 new
0.94% 1273.92% -0.30% 798.09% -28.33% 615.87% %ch from new
65536 954037 972699 981940 65686 973104 260029 1004607
962991 65821 65808 101667 364341 131070
1.01% 1391.83% -0.19% 857.15% -28.63% 666.47%
131072 2039137 2101881 2091491 131232 2092894 554790 2161379
2057533 131410 131361 206193 728871 262142
2.16% 1491.58% -0.10% 915.02% -23.88% 724.51%
262144 4340409 4464460 4403233 262314 4445884 1107842 4584560
4377402 262437 262459 416347 1457945 524286
1.99% 1577.82% -0.06% 967.83% -24.01% 774.44%
524288 9205096 9453356 9408463 524468 9441930 2218577 9692015
9278734 524580 524633 837947 2916107 1048574
1.88% 1693.52% -0.03% 1026.79% -23.92% 824.30%
1048576 19458756 19950272 19838588 1048766 19912134 4430649 20434212
19606028 1048958 1048941 1694896 5832445 2097150
1.76% 1791.27% -0.02% 1074.83% -24.03% 874.38%
Discussion of cases:
*sort: There's no structure in random data to exploit, so the theoretical
limit is lg(n!). Both methods get close to that, and timsort is hugging
it (indeed, in a *marginal* sense, it's a spectacular improvement --
there's only about 1% left before hitting the wall, and timsort knows
darned well it's doing compares that won't pay on random data -- but so
does the samplesort hybrid). For contrast, Hoare's original random-pivot
quicksort does about 39% more compares than the limit, and the median-of-3
variant about 19% more.
3sort, %sort, and !sort: No contest; there's structure in this data, but
not of the specific kinds samplesort special-cases. Note that structure
in !sort wasn't put there on purpose -- it was crafted as a worst case for
a previous quicksort implementation. That timsort nails it came as a
surprise to me (although it's obvious in retrospect).
+sort: samplesort special-cases this data, and does a few less compares
than timsort. However, timsort runs this case significantly faster on all
boxes we have timings for, because timsort is in the business of merging
runs efficiently, while samplesort does much more data movement in this
(for it) special case.
~sort: samplesort's special cases for large masses of equal elements are
extremely effective on ~sort's specific data pattern, and timsort just
isn't going to get close to that, despite that it's clearly getting a
great deal of benefit out of the duplicates (the # of compares is much less
than lg(n!)). ~sort has a perfectly uniform distribution of just 4
distinct values, and as the distribution gets more skewed, samplesort's
equal-element gimmicks become less effective, while timsort's adaptive
strategies find more to exploit; in a database supplied by Kevin Altis, a
sort on its highly skewed "on which stock exchange does this company's
stock trade?" field ran over twice as fast under timsort.
However, despite that timsort does many more comparisons on ~sort, and
that on several platforms ~sort runs highly significantly slower under
timsort, on other platforms ~sort runs highly significantly faster under
timsort. No other kind of data has shown this wild x-platform behavior,
and we don't have an explanation for it. The only thing I can think of
that could transform what "should be" highly significant slowdowns into
highly significant speedups on some boxes are catastrophic cache effects
in samplesort.
But timsort "should be" slower than samplesort on ~sort, so it's hard
to count that it isn't on some boxes as a strike against it <wink>.
+ Here's the highwater mark for the number of heap-based temp slots (4
bytes each on this box) needed by each test, again with arguments
"15 20 1":
2**i *sort \sort /sort 3sort +sort %sort ~sort =sort !sort
32768 16384 0 0 6256 0 10821 12288 0 16383
65536 32766 0 0 21652 0 31276 24576 0 32767
131072 65534 0 0 17258 0 58112 49152 0 65535
262144 131072 0 0 35660 0 123561 98304 0 131071
524288 262142 0 0 31302 0 212057 196608 0 262143
1048576 524286 0 0 312438 0 484942 393216 0 524287
Discussion: The tests that end up doing (close to) perfectly balanced
merges (*sort, !sort) need all N//2 temp slots (or almost all). ~sort
also ends up doing balanced merges, but systematically benefits a lot from
the preliminary pre-merge searches described under "Merge Memory" later.
%sort approaches having a balanced merge at the end because the random
selection of elements to replace is expected to produce an out-of-order
element near the midpoint. \sort, /sort, =sort are the trivial one-run
cases, needing no merging at all. +sort ends up having one very long run
and one very short, and so gets all the temp space it needs from the small
temparray member of the MergeState struct (note that the same would be
true if the new random elements were prefixed to the sorted list instead,
but not if they appeared "in the middle"). 3sort approaches N//3 temp
slots twice, but the run lengths that remain after 3 random exchanges
clearly has very high variance.
A detailed description of timsort follows.
Runs
----
count_run() returns the # of elements in the next run. A run is either
"ascending", which means non-decreasing:
a0 <= a1 <= a2 <= ...
or "descending", which means strictly decreasing:
a0 > a1 > a2 > ...
Note that a run is always at least 2 long, unless we start at the array's
last element.
The definition of descending is strict, because the main routine reverses
a descending run in-place, transforming a descending run into an ascending
run. Reversal is done via the obvious fast "swap elements starting at each
end, and converge at the middle" method, and that can violate stability if
the slice contains any equal elements. Using a strict definition of
descending ensures that a descending run contains distinct elements.
If an array is random, it's very unlikely we'll see long runs. If a natural
run contains less than minrun elements (see next section), the main loop
artificially boosts it to minrun elements, via a stable binary insertion sort
applied to the right number of array elements following the short natural
run. In a random array, *all* runs are likely to be minrun long as a
result. This has two primary good effects:
1. Random data strongly tends then toward perfectly balanced (both runs have
the same length) merges, which is the most efficient way to proceed when
data is random.
2. Because runs are never very short, the rest of the code doesn't make
heroic efforts to shave a few cycles off per-merge overheads. For
example, reasonable use of function calls is made, rather than trying to
inline everything. Since there are no more than N/minrun runs to begin
with, a few "extra" function calls per merge is barely measurable.
Computing minrun
----------------
If N < 64, minrun is N. IOW, binary insertion sort is used for the whole
array then; it's hard to beat that given the overheads of trying something
fancier (see note BINSORT).
When N is a power of 2, testing on random data showed that minrun values of
16, 32, 64 and 128 worked about equally well. At 256 the data-movement cost
in binary insertion sort clearly hurt, and at 8 the increase in the number
of function calls clearly hurt. Picking *some* power of 2 is important
here, so that the merges end up perfectly balanced (see next section). We
pick 32 as a good value in the sweet range; picking a value at the low end
allows the adaptive gimmicks more opportunity to exploit shorter natural
runs.
Because sortperf.py only tries powers of 2, it took a long time to notice
that 32 isn't a good choice for the general case! Consider N=2112:
>>> divmod(2112, 32)
(66, 0)
>>>
If the data is randomly ordered, we're very likely to end up with 66 runs
each of length 32. The first 64 of these trigger a sequence of perfectly
balanced merges (see next section), leaving runs of lengths 2048 and 64 to
merge at the end. The adaptive gimmicks can do that with fewer than 2048+64
compares, but it's still more compares than necessary, and-- mergesort's
bugaboo relative to samplesort --a lot more data movement (O(N) copies just
to get 64 elements into place).
If we take minrun=33 in this case, then we're very likely to end up with 64
runs each of length 33, and then all merges are perfectly balanced. Better!
What we want to avoid is picking minrun such that in
q, r = divmod(N, minrun)
q is a power of 2 and r>0 (then the last merge only gets r elements into
place, and r < minrun is small compared to N), or q a little larger than a
power of 2 regardless of r (then we've got a case similar to "2112", again
leaving too little work for the last merge to do).
Instead we pick a minrun in range(32, 65) such that N/minrun is exactly a
power of 2, or if that isn't possible, is close to, but strictly less than,
a power of 2. This is easier to do than it may sound: take the first 6
bits of N, and add 1 if any of the remaining bits are set. In fact, that
rule covers every case in this section, including small N and exact powers
of 2; merge_compute_minrun() is a deceptively simple function.
The Merge Pattern
-----------------
In order to exploit regularities in the data, we're merging on natural
run lengths, and they can become wildly unbalanced. That's a Good Thing
for this sort! It means we have to find a way to manage an assortment of
potentially very different run lengths, though.
Stability constrains permissible merging patterns. For example, if we have
3 consecutive runs of lengths
A:10000 B:20000 C:10000
we dare not merge A with C first, because if A, B and C happen to contain
a common element, it would get out of order wrt its occurrence(s) in B. The
merging must be done as (A+B)+C or A+(B+C) instead.
So merging is always done on two consecutive runs at a time, and in-place,
although this may require some temp memory (more on that later).
When a run is identified, its base address and length are pushed on a stack
in the MergeState struct. merge_collapse() is then called to see whether it
should merge it with preceding run(s). We would like to delay merging as
long as possible in order to exploit patterns that may come up later, but we
like even more to do merging as soon as possible to exploit that the run just
found is still high in the memory hierarchy. We also can't delay merging
"too long" because it consumes memory to remember the runs that are still
unmerged, and the stack has a fixed size.
What turned out to be a good compromise maintains two invariants on the
stack entries, where A, B and C are the lengths of the three righmost not-yet
merged slices:
1. A > B+C
2. B > C
Note that, by induction, #2 implies the lengths of pending runs form a
decreasing sequence. #1 implies that, reading the lengths right to left,
the pending-run lengths grow at least as fast as the Fibonacci numbers.
Therefore the stack can never grow larger than about log_base_phi(N) entries,
where phi = (1+sqrt(5))/2 ~= 1.618. Thus a small # of stack slots suffice
for very large arrays.
If A <= B+C, the smaller of A and C is merged with B (ties favor C, for the
freshness-in-cache reason), and the new run replaces the A,B or B,C entries;
e.g., if the last 3 entries are
A:30 B:20 C:10
then B is merged with C, leaving
A:30 BC:30
on the stack. Or if they were
A:500 B:400: C:1000
then A is merged with B, leaving
AB:900 C:1000
on the stack.
In both examples, the stack configuration after the merge still violates
invariant #2, and merge_collapse() goes on to continue merging runs until
both invariants are satisfied. As an extreme case, suppose we didn't do the
minrun gimmick, and natural runs were of lengths 128, 64, 32, 16, 8, 4, 2,
and 2. Nothing would get merged until the final 2 was seen, and that would
trigger 7 perfectly balanced merges.
The thrust of these rules when they trigger merging is to balance the run
lengths as closely as possible, while keeping a low bound on the number of
runs we have to remember. This is maximally effective for random data,
where all runs are likely to be of (artificially forced) length minrun, and
then we get a sequence of perfectly balanced merges (with, perhaps, some
oddballs at the end).
OTOH, one reason this sort is so good for partly ordered data has to do
with wildly unbalanced run lengths.
Merge Memory
------------
Merging adjacent runs of lengths A and B in-place, and in linear time, is
difficult. Theoretical constructions are known that can do it, but they're
too difficult and slow for practical use. But if we have temp memory equal
to min(A, B), it's easy.
If A is smaller (function merge_lo), copy A to a temp array, leave B alone,
and then we can do the obvious merge algorithm left to right, from the temp
area and B, starting the stores into where A used to live. There's always a
free area in the original area comprising a number of elements equal to the
number not yet merged from the temp array (trivially true at the start;
proceed by induction). The only tricky bit is that if a comparison raises an
exception, we have to remember to copy the remaining elements back in from
the temp area, lest the array end up with duplicate entries from B. But
that's exactly the same thing we need to do if we reach the end of B first,
so the exit code is pleasantly common to both the normal and error cases.
If B is smaller (function merge_hi, which is merge_lo's "mirror image"),
much the same, except that we need to merge right to left, copying B into a
temp array and starting the stores at the right end of where B used to live.
A refinement: When we're about to merge adjacent runs A and B, we first do
a form of binary search (more on that later) to see where B[0] should end up
in A. Elements in A preceding that point are already in their final
positions, effectively shrinking the size of A. Likewise we also search to
see where A[-1] should end up in B, and elements of B after that point can
also be ignored. This cuts the amount of temp memory needed by the same
amount.
These preliminary searches may not pay off, and can be expected *not* to
repay their cost if the data is random. But they can win huge in all of
time, copying, and memory savings when they do pay, so this is one of the
"per-merge overheads" mentioned above that we're happy to endure because
there is at most one very short run. It's generally true in this algorithm
that we're willing to gamble a little to win a lot, even though the net
expectation is negative for random data.
Merge Algorithms
----------------
merge_lo() and merge_hi() are where the bulk of the time is spent. merge_lo
deals with runs where A <= B, and merge_hi where A > B. They don't know
whether the data is clustered or uniform, but a lovely thing about merging
is that many kinds of clustering "reveal themselves" by how many times in a
row the winning merge element comes from the same run. We'll only discuss
merge_lo here; merge_hi is exactly analogous.
Merging begins in the usual, obvious way, comparing the first element of A
to the first of B, and moving B[0] to the merge area if it's less than A[0],
else moving A[0] to the merge area. Call that the "one pair at a time"
mode. The only twist here is keeping track of how many times in a row "the
winner" comes from the same run.
If that count reaches MIN_GALLOP, we switch to "galloping mode". Here
we *search* B for where A[0] belongs, and move over all the B's before
that point in one chunk to the merge area, then move A[0] to the merge
area. Then we search A for where B[0] belongs, and similarly move a
slice of A in one chunk. Then back to searching B for where A[0] belongs,
etc. We stay in galloping mode until both searches find slices to copy
less than MIN_GALLOP elements long, at which point we go back to one-pair-
at-a-time mode.
A refinement: The MergeState struct contains the value of min_gallop that
controls when we enter galloping mode, initialized to MIN_GALLOP.
merge_lo() and merge_hi() adjust this higher when galloping isn't paying
off, and lower when it is.
Galloping
---------
Still without loss of generality, assume A is the shorter run. In galloping
mode, we first look for A[0] in B. We do this via "galloping", comparing
A[0] in turn to B[0], B[1], B[3], B[7], ..., B[2**j - 1], ..., until finding
the k such that B[2**(k-1) - 1] < A[0] <= B[2**k - 1]. This takes at most
roughly lg(B) comparisons, and, unlike a straight binary search, favors
finding the right spot early in B (more on that later).
After finding such a k, the region of uncertainty is reduced to 2**(k-1) - 1
consecutive elements, and a straight binary search requires exactly k-1
additional comparisons to nail it (see note REGION OF UNCERTAINTY). Then we
copy all the B's up to that point in one chunk, and then copy A[0]. Note
that no matter where A[0] belongs in B, the combination of galloping + binary
search finds it in no more than about 2*lg(B) comparisons.
If we did a straight binary search, we could find it in no more than
ceiling(lg(B+1)) comparisons -- but straight binary search takes that many
comparisons no matter where A[0] belongs. Straight binary search thus loses
to galloping unless the run is quite long, and we simply can't guess
whether it is in advance.
If data is random and runs have the same length, A[0] belongs at B[0] half
the time, at B[1] a quarter of the time, and so on: a consecutive winning
sub-run in B of length k occurs with probability 1/2**(k+1). So long
winning sub-runs are extremely unlikely in random data, and guessing that a
winning sub-run is going to be long is a dangerous game.
OTOH, if data is lopsided or lumpy or contains many duplicates, long
stretches of winning sub-runs are very likely, and cutting the number of
comparisons needed to find one from O(B) to O(log B) is a huge win.
Galloping compromises by getting out fast if there isn't a long winning
sub-run, yet finding such very efficiently when they exist.
I first learned about the galloping strategy in a related context; see:
"Adaptive Set Intersections, Unions, and Differences" (2000)
Erik D. Demaine, Alejandro L<>pez-Ortiz, J. Ian Munro
and its followup(s). An earlier paper called the same strategy
"exponential search":
"Optimistic Sorting and Information Theoretic Complexity"
Peter McIlroy
SODA (Fourth Annual ACM-SIAM Symposium on Discrete Algorithms), pp
467-474, Austin, Texas, 25-27 January 1993.
and it probably dates back to an earlier paper by Bentley and Yao. The
McIlroy paper in particular has good analysis of a mergesort that's
probably strongly related to this one in its galloping strategy.
Galloping with a Broken Leg
---------------------------
So why don't we always gallop? Because it can lose, on two counts:
1. While we're willing to endure small per-merge overheads, per-comparison
overheads are a different story. Calling Yet Another Function per
comparison is expensive, and gallop_left() and gallop_right() are
too long-winded for sane inlining.
2. Galloping can-- alas --require more comparisons than linear one-at-time
search, depending on the data.
#2 requires details. If A[0] belongs before B[0], galloping requires 1
compare to determine that, same as linear search, except it costs more
to call the gallop function. If A[0] belongs right before B[1], galloping
requires 2 compares, again same as linear search. On the third compare,
galloping checks A[0] against B[3], and if it's <=, requires one more
compare to determine whether A[0] belongs at B[2] or B[3]. That's a total
of 4 compares, but if A[0] does belong at B[2], linear search would have
discovered that in only 3 compares, and that's a huge loss! Really. It's
an increase of 33% in the number of compares needed, and comparisons are
expensive in Python.
index in B where # compares linear # gallop # binary gallop
A[0] belongs search needs compares compares total
---------------- ----------------- -------- -------- ------
0 1 1 0 1
1 2 2 0 2
2 3 3 1 4
3 4 3 1 4
4 5 4 2 6
5 6 4 2 6
6 7 4 2 6
7 8 4 2 6
8 9 5 3 8
9 10 5 3 8
10 11 5 3 8
11 12 5 3 8
...
In general, if A[0] belongs at B[i], linear search requires i+1 comparisons
to determine that, and galloping a total of 2*floor(lg(i))+2 comparisons.
The advantage of galloping is unbounded as i grows, but it doesn't win at
all until i=6. Before then, it loses twice (at i=2 and i=4), and ties
at the other values. At and after i=6, galloping always wins.
We can't guess in advance when it's going to win, though, so we do one pair
at a time until the evidence seems strong that galloping may pay. MIN_GALLOP
is 7, and that's pretty strong evidence. However, if the data is random, it
simply will trigger galloping mode purely by luck every now and again, and
it's quite likely to hit one of the losing cases next. On the other hand,
in cases like ~sort, galloping always pays, and MIN_GALLOP is larger than it
"should be" then. So the MergeState struct keeps a min_gallop variable
that merge_lo and merge_hi adjust: the longer we stay in galloping mode,
the smaller min_gallop gets, making it easier to transition back to
galloping mode (if we ever leave it in the current merge, and at the
start of the next merge). But whenever the gallop loop doesn't pay,
min_gallop is increased by one, making it harder to transition back
to galloping mode (and again both within a merge and across merges). For
random data, this all but eliminates the gallop penalty: min_gallop grows
large enough that we almost never get into galloping mode. And for cases
like ~sort, min_gallop can fall to as low as 1. This seems to work well,
but in all it's a minor improvement over using a fixed MIN_GALLOP value.
Galloping Complication
----------------------
The description above was for merge_lo. merge_hi has to merge "from the
other end", and really needs to gallop starting at the last element in a run
instead of the first. Galloping from the first still works, but does more
comparisons than it should (this is significant -- I timed it both ways). For
this reason, the gallop_left() and gallop_right() (see note LEFT OR RIGHT)
functions have a "hint" argument, which is the index at which galloping
should begin. So galloping can actually start at any index, and proceed at
offsets of 1, 3, 7, 15, ... or -1, -3, -7, -15, ... from the starting index.
In the code as I type it's always called with either 0 or n-1 (where n is
the # of elements in a run). It's tempting to try to do something fancier,
melding galloping with some form of interpolation search; for example, if
we're merging a run of length 1 with a run of length 10000, index 5000 is
probably a better guess at the final result than either 0 or 9999. But
it's unclear how to generalize that intuition usefully, and merging of
wildly unbalanced runs already enjoys excellent performance.
~sort is a good example of when balanced runs could benefit from a better
hint value: to the extent possible, this would like to use a starting
offset equal to the previous value of acount/bcount. Doing so saves about
10% of the compares in ~sort. However, doing so is also a mixed bag,
hurting other cases.
Comparing Average # of Compares on Random Arrays
------------------------------------------------
[NOTE: This was done when the new algorithm used about 0.1% more compares
on random data than does its current incarnation.]
Here list.sort() is samplesort, and list.msort() this sort:
"""
import random
from time import clock as now
def fill(n):
from random import random
return [random() for i in xrange(n)]
def mycmp(x, y):
global ncmp
ncmp += 1
return cmp(x, y)
def timeit(values, method):
global ncmp
X = values[:]
bound = getattr(X, method)
ncmp = 0
t1 = now()
bound(mycmp)
t2 = now()
return t2-t1, ncmp
format = "%5s %9.2f %11d"
f2 = "%5s %9.2f %11.2f"
def drive():
count = sst = sscmp = mst = mscmp = nelts = 0
while True:
n = random.randrange(100000)
nelts += n
x = fill(n)
t, c = timeit(x, 'sort')
sst += t
sscmp += c
t, c = timeit(x, 'msort')
mst += t
mscmp += c
count += 1
if count % 10:
continue
print "count", count, "nelts", nelts
print format % ("sort", sst, sscmp)
print format % ("msort", mst, mscmp)
print f2 % ("", (sst-mst)*1e2/mst, (sscmp-mscmp)*1e2/mscmp)
drive()
"""
I ran this on Windows and kept using the computer lightly while it was
running. time.clock() is wall-clock time on Windows, with better than
microsecond resolution. samplesort started with a 1.52% #-of-comparisons
disadvantage, fell quickly to 1.48%, and then fluctuated within that small
range. Here's the last chunk of output before I killed the job:
count 2630 nelts 130906543
sort 6110.80 1937887573
msort 6002.78 1909389381
1.80 1.49
We've done nearly 2 billion comparisons apiece at Python speed there, and
that's enough <wink>.
For random arrays of size 2 (yes, there are only 2 interesting ones),
samplesort has a 50%(!) comparison disadvantage. This is a consequence of
samplesort special-casing at most one ascending run at the start, then
falling back to the general case if it doesn't find an ascending run
immediately. The consequence is that it ends up using two compares to sort
[2, 1]. Gratifyingly, timsort doesn't do any special-casing, so had to be
taught how to deal with mixtures of ascending and descending runs
efficiently in all cases.
NOTES
-----
BINSORT
A "binary insertion sort" is just like a textbook insertion sort, but instead
of locating the correct position of the next item via linear (one at a time)
search, an equivalent to Python's bisect.bisect_right is used to find the
correct position in logarithmic time. Most texts don't mention this
variation, and those that do usually say it's not worth the bother: insertion
sort remains quadratic (expected and worst cases) either way. Speeding the
search doesn't reduce the quadratic data movement costs.
But in CPython's case, comparisons are extraordinarily expensive compared to
moving data, and the details matter. Moving objects is just copying
pointers. Comparisons can be arbitrarily expensive (can invoke arbitary
user-supplied Python code), but even in simple cases (like 3 < 4) _all_
decisions are made at runtime: what's the type of the left comparand? the
type of the right? do they need to be coerced to a common type? where's the
code to compare these types? And so on. Even the simplest Python comparison
triggers a large pile of C-level pointer dereferences, conditionals, and
function calls.
So cutting the number of compares is almost always measurably helpful in
CPython, and the savings swamp the quadratic-time data movement costs for
reasonable minrun values.
LEFT OR RIGHT
gallop_left() and gallop_right() are akin to the Python bisect module's
bisect_left() and bisect_right(): they're the same unless the slice they're
searching contains a (at least one) value equal to the value being searched
for. In that case, gallop_left() returns the position immediately before the
leftmost equal value, and gallop_right() the position immediately after the
rightmost equal value. The distinction is needed to preserve stability. In
general, when merging adjacent runs A and B, gallop_left is used to search
thru B for where an element from A belongs, and gallop_right to search thru A
for where an element from B belongs.
REGION OF UNCERTAINTY
Two kinds of confusion seem to be common about the claim that after finding
a k such that
B[2**(k-1) - 1] < A[0] <= B[2**k - 1]
then a binary search requires exactly k-1 tries to find A[0]'s proper
location. For concreteness, say k=3, so B[3] < A[0] <= B[7].
The first confusion takes the form "OK, then the region of uncertainty is at
indices 3, 4, 5, 6 and 7: that's 5 elements, not the claimed 2**(k-1) - 1 =
3"; or the region is viewed as a Python slice and the objection is "but that's
the slice B[3:7], so has 7-3 = 4 elements". Resolution: we've already
compared A[0] against B[3] and against B[7], so A[0]'s correct location is
already known wrt _both_ endpoints. What remains is to find A[0]'s correct
location wrt B[4], B[5] and B[6], which spans 3 elements. Or in general, the
slice (leaving off both endpoints) (2**(k-1)-1)+1 through (2**k-1)-1
inclusive = 2**(k-1) through (2**k-1)-1 inclusive, which has
(2**k-1)-1 - 2**(k-1) + 1 =
2**k-1 - 2**(k-1) =
2*2**k-1 - 2**(k-1) =
(2-1)*2**(k-1) - 1 =
2**(k-1) - 1
elements.
The second confusion: "k-1 = 2 binary searches can find the correct location
among 2**(k-1) = 4 elements, but you're only applying it to 3 elements: we
could make this more efficient by arranging for the region of uncertainty to
span 2**(k-1) elements." Resolution: that confuses "elements" with
"locations". In a slice with N elements, there are N+1 _locations_. In the
example, with the region of uncertainty B[4], B[5], B[6], there are 4
locations: before B[4], between B[4] and B[5], between B[5] and B[6], and
after B[6]. In general, across 2**(k-1)-1 elements, there are 2**(k-1)
locations. That's why k-1 binary searches are necessary and sufficient.

View File

@ -0,0 +1,124 @@
All about co_lnotab, the line number table.
Code objects store a field named co_lnotab. This is an array of unsigned bytes
disguised as a Python string. It is used to map bytecode offsets to source code
line #s for tracebacks and to identify line number boundaries for line tracing.
The array is conceptually a compressed list of
(bytecode offset increment, line number increment)
pairs. The details are important and delicate, best illustrated by example:
byte code offset source code line number
0 1
6 2
50 7
350 307
361 308
Instead of storing these numbers literally, we compress the list by storing only
the increments from one row to the next. Conceptually, the stored list might
look like:
0, 1, 6, 1, 44, 5, 300, 300, 11, 1
The above doesn't really work, but it's a start. Note that an unsigned byte
can't hold negative values, or values larger than 255, and the above example
contains two such values. So we make two tweaks:
(a) there's a deep assumption that byte code offsets and their corresponding
line #s both increase monotonically, and
(b) if at least one column jumps by more than 255 from one row to the next,
more than one pair is written to the table. In case #b, there's no way to know
from looking at the table later how many were written. That's the delicate
part. A user of co_lnotab desiring to find the source line number
corresponding to a bytecode address A should do something like this
lineno = addr = 0
for addr_incr, line_incr in co_lnotab:
addr += addr_incr
if addr > A:
return lineno
lineno += line_incr
(In C, this is implemented by PyCode_Addr2Line().) In order for this to work,
when the addr field increments by more than 255, the line # increment in each
pair generated must be 0 until the remaining addr increment is < 256. So, in
the example above, assemble_lnotab in compile.c should not (as was actually done
until 2.2) expand 300, 300 to
255, 255, 45, 45,
but to
255, 0, 45, 255, 0, 45.
The above is sufficient to reconstruct line numbers for tracebacks, but not for
line tracing. Tracing is handled by PyCode_CheckLineNumber() in codeobject.c
and maybe_call_line_trace() in ceval.c.
*** Tracing ***
To a first approximation, we want to call the tracing function when the line
number of the current instruction changes. Re-computing the current line for
every instruction is a little slow, though, so each time we compute the line
number we save the bytecode indices where it's valid:
*instr_lb <= frame->f_lasti < *instr_ub
is true so long as execution does not change lines. That is, *instr_lb holds
the first bytecode index of the current line, and *instr_ub holds the first
bytecode index of the next line. As long as the above expression is true,
maybe_call_line_trace() does not need to call PyCode_CheckLineNumber(). Note
that the same line may appear multiple times in the lnotab, either because the
bytecode jumped more than 255 indices between line number changes or because
the compiler inserted the same line twice. Even in that case, *instr_ub holds
the first index of the next line.
However, we don't *always* want to call the line trace function when the above
test fails.
Consider this code:
1: def f(a):
2: while a:
3: print 1,
4: break
5: else:
6: print 2,
which compiles to this:
2 0 SETUP_LOOP 19 (to 22)
>> 3 LOAD_FAST 0 (a)
6 POP_JUMP_IF_FALSE 17
3 9 LOAD_CONST 1 (1)
12 PRINT_ITEM
4 13 BREAK_LOOP
14 JUMP_ABSOLUTE 3
>> 17 POP_BLOCK
6 18 LOAD_CONST 2 (2)
21 PRINT_ITEM
>> 22 LOAD_CONST 0 (None)
25 RETURN_VALUE
If 'a' is false, execution will jump to the POP_BLOCK instruction at offset 17
and the co_lnotab will claim that execution has moved to line 4, which is wrong.
In this case, we could instead associate the POP_BLOCK with line 5, but that
would break jumps around loops without else clauses.
We fix this by only calling the line trace function for a forward jump if the
co_lnotab indicates we have jumped to the *start* of a line, i.e. if the current
instruction offset matches the offset given for the start of a line by the
co_lnotab. For backward jumps, however, we always call the line trace function,
which lets a debugger stop on every evaluation of a loop guard (which usually
won't be the first opcode in a line).
Why do we set f_lineno when tracing, and only just before calling the trace
function? Well, consider the code above when 'a' is true. If stepping through
this with 'n' in pdb, you would stop at line 1 with a "call" type event, then
line events on lines 2, 3, and 4, then a "return" type event -- but because the
code for the return actually falls in the range of the "line 6" opcodes, you
would be shown line 6 during this event. This is a change from the behaviour in
2.2 and before, and I've found it confusing in practice. By setting and using
f_lineno when tracing, one can report a line number different from that
suggested by f_lasti on this one occasion where it's desirable.

View File

@ -0,0 +1,842 @@
/* Memoryview object implementation */
#include "Python.h"
static Py_ssize_t
get_shape0(Py_buffer *buf)
{
if (buf->shape != NULL)
return buf->shape[0];
if (buf->ndim == 0)
return 1;
PyErr_SetString(PyExc_TypeError,
"exported buffer does not have any shape information associated "
"to it");
return -1;
}
static void
dup_buffer(Py_buffer *dest, Py_buffer *src)
{
*dest = *src;
if (src->ndim == 1 && src->shape != NULL) {
dest->shape = &(dest->smalltable[0]);
dest->shape[0] = get_shape0(src);
}
if (src->ndim == 1 && src->strides != NULL) {
dest->strides = &(dest->smalltable[1]);
dest->strides[0] = src->strides[0];
}
}
static int
memory_getbuf(PyMemoryViewObject *self, Py_buffer *view, int flags)
{
int res = 0;
if (self->view.obj != NULL)
res = PyObject_GetBuffer(self->view.obj, view, flags);
if (view)
dup_buffer(view, &self->view);
return res;
}
static void
memory_releasebuf(PyMemoryViewObject *self, Py_buffer *view)
{
PyBuffer_Release(view);
}
PyDoc_STRVAR(memory_doc,
"memoryview(object)\n\
\n\
Create a new memoryview object which references the given object.");
PyObject *
PyMemoryView_FromBuffer(Py_buffer *info)
{
PyMemoryViewObject *mview;
mview = (PyMemoryViewObject *)
PyObject_GC_New(PyMemoryViewObject, &PyMemoryView_Type);
if (mview == NULL)
return NULL;
mview->base = NULL;
dup_buffer(&mview->view, info);
/* NOTE: mview->view.obj should already have been incref'ed as
part of PyBuffer_FillInfo(). */
_PyObject_GC_TRACK(mview);
return (PyObject *)mview;
}
PyObject *
PyMemoryView_FromObject(PyObject *base)
{
PyMemoryViewObject *mview;
Py_buffer view;
if (!PyObject_CheckBuffer(base)) {
PyErr_SetString(PyExc_TypeError,
"cannot make memory view because object does "
"not have the buffer interface");
return NULL;
}
if (PyObject_GetBuffer(base, &view, PyBUF_FULL_RO) < 0)
return NULL;
mview = (PyMemoryViewObject *)PyMemoryView_FromBuffer(&view);
if (mview == NULL) {
PyBuffer_Release(&view);
return NULL;
}
mview->base = base;
Py_INCREF(base);
return (PyObject *)mview;
}
static PyObject *
memory_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds)
{
PyObject *obj;
static char *kwlist[] = {"object", 0};
if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:memoryview", kwlist,
&obj)) {
return NULL;
}
return PyMemoryView_FromObject(obj);
}
static void
_strided_copy_nd(char *dest, char *src, int nd, Py_ssize_t *shape,
Py_ssize_t *strides, Py_ssize_t itemsize, char fort)
{
int k;
Py_ssize_t outstride;
if (nd==0) {
memcpy(dest, src, itemsize);
}
else if (nd == 1) {
for (k = 0; k<shape[0]; k++) {
memcpy(dest, src, itemsize);
dest += itemsize;
src += strides[0];
}
}
else {
if (fort == 'F') {
/* Copy first dimension first,
second dimension second, etc...
Set up the recursive loop backwards so that final
dimension is actually copied last.
*/
outstride = itemsize;
for (k=1; k<nd-1;k++) {
outstride *= shape[k];
}
for (k=0; k<shape[nd-1]; k++) {
_strided_copy_nd(dest, src, nd-1, shape,
strides, itemsize, fort);
dest += outstride;
src += strides[nd-1];
}
}
else {
/* Copy last dimension first,
second-to-last dimension second, etc.
Set up the recursion so that the
first dimension is copied last
*/
outstride = itemsize;
for (k=1; k < nd; k++) {
outstride *= shape[k];
}
for (k=0; k<shape[0]; k++) {
_strided_copy_nd(dest, src, nd-1, shape+1,
strides+1, itemsize,
fort);
dest += outstride;
src += strides[0];
}
}
}
return;
}
static int
_indirect_copy_nd(char *dest, Py_buffer *view, char fort)
{
Py_ssize_t *indices;
int k;
Py_ssize_t elements;
char *ptr;
void (*func)(int, Py_ssize_t *, const Py_ssize_t *);
if (view->ndim > PY_SSIZE_T_MAX / sizeof(Py_ssize_t)) {
PyErr_NoMemory();
return -1;
}
indices = (Py_ssize_t *)PyMem_Malloc(sizeof(Py_ssize_t)*view->ndim);
if (indices == NULL) {
PyErr_NoMemory();
return -1;
}
for (k=0; k<view->ndim;k++) {
indices[k] = 0;
}
elements = 1;
for (k=0; k<view->ndim; k++) {
elements *= view->shape[k];
}
if (fort == 'F') {
func = _Py_add_one_to_index_F;
}
else {
func = _Py_add_one_to_index_C;
}
while (elements--) {
func(view->ndim, indices, view->shape);
ptr = PyBuffer_GetPointer(view, indices);
memcpy(dest, ptr, view->itemsize);
dest += view->itemsize;
}
PyMem_Free(indices);
return 0;
}
/*
Get a the data from an object as a contiguous chunk of memory (in
either 'C' or 'F'ortran order) even if it means copying it into a
separate memory area.
Returns a new reference to a Memory view object. If no copy is needed,
the memory view object points to the original memory and holds a
lock on the original. If a copy is needed, then the memory view object
points to a brand-new Bytes object (and holds a memory lock on it).
buffertype
PyBUF_READ buffer only needs to be read-only
PyBUF_WRITE buffer needs to be writable (give error if not contiguous)
PyBUF_SHADOW buffer needs to be writable so shadow it with
a contiguous buffer if it is not. The view will point to
the shadow buffer which can be written to and then
will be copied back into the other buffer when the memory
view is de-allocated. While the shadow buffer is
being used, it will have an exclusive write lock on
the original buffer.
*/
PyObject *
PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char fort)
{
PyMemoryViewObject *mem;
PyObject *bytes;
Py_buffer *view;
int flags;
char *dest;
if (!PyObject_CheckBuffer(obj)) {
PyErr_SetString(PyExc_TypeError,
"object does not have the buffer interface");
return NULL;
}
mem = PyObject_GC_New(PyMemoryViewObject, &PyMemoryView_Type);
if (mem == NULL)
return NULL;
view = &mem->view;
flags = PyBUF_FULL_RO;
switch(buffertype) {
case PyBUF_WRITE:
flags = PyBUF_FULL;
break;
}
if (PyObject_GetBuffer(obj, view, flags) != 0) {
Py_DECREF(mem);
return NULL;
}
if (PyBuffer_IsContiguous(view, fort)) {
/* no copy needed */
Py_INCREF(obj);
mem->base = obj;
_PyObject_GC_TRACK(mem);
return (PyObject *)mem;
}
/* otherwise a copy is needed */
if (buffertype == PyBUF_WRITE) {
Py_DECREF(mem);
PyErr_SetString(PyExc_BufferError,
"writable contiguous buffer requested "
"for a non-contiguousobject.");
return NULL;
}
bytes = PyBytes_FromStringAndSize(NULL, view->len);
if (bytes == NULL) {
Py_DECREF(mem);
return NULL;
}
dest = PyBytes_AS_STRING(bytes);
/* different copying strategy depending on whether
or not any pointer de-referencing is needed
*/
/* strided or in-direct copy */
if (view->suboffsets==NULL) {
_strided_copy_nd(dest, view->buf, view->ndim, view->shape,
view->strides, view->itemsize, fort);
}
else {
if (_indirect_copy_nd(dest, view, fort) < 0) {
Py_DECREF(bytes);
Py_DECREF(mem);
return NULL;
}
}
if (buffertype == PyBUF_SHADOW) {
/* return a shadowed memory-view object */
view->buf = dest;
mem->base = PyTuple_Pack(2, obj, bytes);
Py_DECREF(bytes);
if (mem->base == NULL) {
Py_DECREF(mem);
return NULL;
}
}
else {
PyBuffer_Release(view); /* XXX ? */
/* steal the reference */
mem->base = bytes;
}
_PyObject_GC_TRACK(mem);
return (PyObject *)mem;
}
static PyObject *
memory_format_get(PyMemoryViewObject *self)
{
return PyString_FromString(self->view.format);
}
static PyObject *
memory_itemsize_get(PyMemoryViewObject *self)
{
return PyLong_FromSsize_t(self->view.itemsize);
}
static PyObject *
_IntTupleFromSsizet(int len, Py_ssize_t *vals)
{
int i;
PyObject *o;
PyObject *intTuple;
if (vals == NULL) {
Py_INCREF(Py_None);
return Py_None;
}
intTuple = PyTuple_New(len);
if (!intTuple) return NULL;
for(i=0; i<len; i++) {
o = PyLong_FromSsize_t(vals[i]);
if (!o) {
Py_DECREF(intTuple);
return NULL;
}
PyTuple_SET_ITEM(intTuple, i, o);
}
return intTuple;
}
static PyObject *
memory_shape_get(PyMemoryViewObject *self)
{
return _IntTupleFromSsizet(self->view.ndim, self->view.shape);
}
static PyObject *
memory_strides_get(PyMemoryViewObject *self)
{
return _IntTupleFromSsizet(self->view.ndim, self->view.strides);
}
static PyObject *
memory_suboffsets_get(PyMemoryViewObject *self)
{
return _IntTupleFromSsizet(self->view.ndim, self->view.suboffsets);
}
static PyObject *
memory_readonly_get(PyMemoryViewObject *self)
{
return PyBool_FromLong(self->view.readonly);
}
static PyObject *
memory_ndim_get(PyMemoryViewObject *self)
{
return PyLong_FromLong(self->view.ndim);
}
static PyGetSetDef memory_getsetlist[] ={
{"format", (getter)memory_format_get, NULL, NULL},
{"itemsize", (getter)memory_itemsize_get, NULL, NULL},
{"shape", (getter)memory_shape_get, NULL, NULL},
{"strides", (getter)memory_strides_get, NULL, NULL},
{"suboffsets", (getter)memory_suboffsets_get, NULL, NULL},
{"readonly", (getter)memory_readonly_get, NULL, NULL},
{"ndim", (getter)memory_ndim_get, NULL, NULL},
{NULL, NULL, NULL, NULL},
};
static PyObject *
memory_tobytes(PyMemoryViewObject *self, PyObject *noargs)
{
Py_buffer view;
PyObject *res;
if (PyObject_GetBuffer((PyObject *)self, &view, PyBUF_SIMPLE) < 0)
return NULL;
res = PyBytes_FromStringAndSize(NULL, view.len);
PyBuffer_ToContiguous(PyBytes_AS_STRING(res), &view, view.len, 'C');
PyBuffer_Release(&view);
return res;
}
/* TODO: rewrite this function using the struct module to unpack
each buffer item */
static PyObject *
memory_tolist(PyMemoryViewObject *mem, PyObject *noargs)
{
Py_buffer *view = &(mem->view);
Py_ssize_t i;
PyObject *res, *item;
char *buf;
if (strcmp(view->format, "B") || view->itemsize != 1) {
PyErr_SetString(PyExc_NotImplementedError,
"tolist() only supports byte views");
return NULL;
}
if (view->ndim != 1) {
PyErr_SetString(PyExc_NotImplementedError,
"tolist() only supports one-dimensional objects");
return NULL;
}
res = PyList_New(view->len);
if (res == NULL)
return NULL;
buf = view->buf;
for (i = 0; i < view->len; i++) {
item = PyInt_FromLong((unsigned char) *buf);
if (item == NULL) {
Py_DECREF(res);
return NULL;
}
PyList_SET_ITEM(res, i, item);
buf++;
}
return res;
}
static PyMethodDef memory_methods[] = {
{"tobytes", (PyCFunction)memory_tobytes, METH_NOARGS, NULL},
{"tolist", (PyCFunction)memory_tolist, METH_NOARGS, NULL},
{NULL, NULL} /* sentinel */
};
static void
memory_dealloc(PyMemoryViewObject *self)
{
_PyObject_GC_UNTRACK(self);
if (self->view.obj != NULL) {
if (self->base && PyTuple_Check(self->base)) {
/* Special case when first element is generic object
with buffer interface and the second element is a
contiguous "shadow" that must be copied back into
the data areay of the first tuple element before
releasing the buffer on the first element.
*/
PyObject_CopyData(PyTuple_GET_ITEM(self->base,0),
PyTuple_GET_ITEM(self->base,1));
/* The view member should have readonly == -1 in
this instance indicating that the memory can
be "locked" and was locked and will be unlocked
again after this call.
*/
PyBuffer_Release(&(self->view));
}
else {
PyBuffer_Release(&(self->view));
}
Py_CLEAR(self->base);
}
PyObject_GC_Del(self);
}
static PyObject *
memory_repr(PyMemoryViewObject *self)
{
return PyString_FromFormat("<memory at %p>", self);
}
/* Sequence methods */
static Py_ssize_t
memory_length(PyMemoryViewObject *self)
{
return get_shape0(&self->view);
}
/* Alternate version of memory_subcript that only accepts indices.
Used by PySeqIter_New().
*/
static PyObject *
memory_item(PyMemoryViewObject *self, Py_ssize_t result)
{
Py_buffer *view = &(self->view);
if (view->ndim == 0) {
PyErr_SetString(PyExc_IndexError,
"invalid indexing of 0-dim memory");
return NULL;
}
if (view->ndim == 1) {
/* Return a bytes object */
char *ptr;
ptr = (char *)view->buf;
if (result < 0) {
result += get_shape0(view);
}
if ((result < 0) || (result >= get_shape0(view))) {
PyErr_SetString(PyExc_IndexError,
"index out of bounds");
return NULL;
}
if (view->strides == NULL)
ptr += view->itemsize * result;
else
ptr += view->strides[0] * result;
if (view->suboffsets != NULL &&
view->suboffsets[0] >= 0) {
ptr = *((char **)ptr) + view->suboffsets[0];
}
return PyBytes_FromStringAndSize(ptr, view->itemsize);
} else {
/* Return a new memory-view object */
Py_buffer newview;
memset(&newview, 0, sizeof(newview));
/* XXX: This needs to be fixed so it actually returns a sub-view */
return PyMemoryView_FromBuffer(&newview);
}
}
/*
mem[obj] returns a bytes object holding the data for one element if
obj fully indexes the memory view or another memory-view object
if it does not.
0-d memory-view objects can be referenced using ... or () but
not with anything else.
*/
static PyObject *
memory_subscript(PyMemoryViewObject *self, PyObject *key)
{
Py_buffer *view;
view = &(self->view);
if (view->ndim == 0) {
if (key == Py_Ellipsis ||
(PyTuple_Check(key) && PyTuple_GET_SIZE(key)==0)) {
Py_INCREF(self);
return (PyObject *)self;
}
else {
PyErr_SetString(PyExc_IndexError,
"invalid indexing of 0-dim memory");
return NULL;
}
}
if (PyIndex_Check(key)) {
Py_ssize_t result;
result = PyNumber_AsSsize_t(key, NULL);
if (result == -1 && PyErr_Occurred())
return NULL;
return memory_item(self, result);
}
else if (PySlice_Check(key)) {
Py_ssize_t start, stop, step, slicelength;
if (PySlice_GetIndicesEx((PySliceObject*)key, get_shape0(view),
&start, &stop, &step, &slicelength) < 0) {
return NULL;
}
if (step == 1 && view->ndim == 1) {
Py_buffer newview;
void *newbuf = (char *) view->buf
+ start * view->itemsize;
int newflags = view->readonly
? PyBUF_CONTIG_RO : PyBUF_CONTIG;
/* XXX There should be an API to create a subbuffer */
if (view->obj != NULL) {
if (PyObject_GetBuffer(view->obj, &newview, newflags) == -1)
return NULL;
}
else {
newview = *view;
}
newview.buf = newbuf;
newview.len = slicelength * newview.itemsize;
newview.format = view->format;
newview.shape = &(newview.smalltable[0]);
newview.shape[0] = slicelength;
newview.strides = &(newview.itemsize);
return PyMemoryView_FromBuffer(&newview);
}
PyErr_SetNone(PyExc_NotImplementedError);
return NULL;
}
PyErr_Format(PyExc_TypeError,
"cannot index memory using \"%.200s\"",
key->ob_type->tp_name);
return NULL;
}
/* Need to support assigning memory if we can */
static int
memory_ass_sub(PyMemoryViewObject *self, PyObject *key, PyObject *value)
{
Py_ssize_t start, len, bytelen;
Py_buffer srcview;
Py_buffer *view = &(self->view);
char *srcbuf, *destbuf;
if (view->readonly) {
PyErr_SetString(PyExc_TypeError,
"cannot modify read-only memory");
return -1;
}
if (value == NULL) {
PyErr_SetString(PyExc_TypeError,
"cannot delete memory");
return -1;
}
if (view->ndim != 1) {
PyErr_SetNone(PyExc_NotImplementedError);
return -1;
}
if (PyIndex_Check(key)) {
start = PyNumber_AsSsize_t(key, NULL);
if (start == -1 && PyErr_Occurred())
return -1;
if (start < 0) {
start += get_shape0(view);
}
if ((start < 0) || (start >= get_shape0(view))) {
PyErr_SetString(PyExc_IndexError,
"index out of bounds");
return -1;
}
len = 1;
}
else if (PySlice_Check(key)) {
Py_ssize_t stop, step;
if (PySlice_GetIndicesEx((PySliceObject*)key, get_shape0(view),
&start, &stop, &step, &len) < 0) {
return -1;
}
if (step != 1) {
PyErr_SetNone(PyExc_NotImplementedError);
return -1;
}
}
else {
PyErr_Format(PyExc_TypeError,
"cannot index memory using \"%.200s\"",
key->ob_type->tp_name);
return -1;
}
if (PyObject_GetBuffer(value, &srcview, PyBUF_CONTIG_RO) == -1) {
return -1;
}
/* XXX should we allow assignment of different item sizes
as long as the byte length is the same?
(e.g. assign 2 shorts to a 4-byte slice) */
if (srcview.itemsize != view->itemsize) {
PyErr_Format(PyExc_TypeError,
"mismatching item sizes for \"%.200s\" and \"%.200s\"",
view->obj->ob_type->tp_name, srcview.obj->ob_type->tp_name);
goto _error;
}
bytelen = len * view->itemsize;
if (bytelen != srcview.len) {
PyErr_SetString(PyExc_ValueError,
"cannot modify size of memoryview object");
goto _error;
}
/* Do the actual copy */
destbuf = (char *) view->buf + start * view->itemsize;
srcbuf = (char *) srcview.buf;
if (destbuf + bytelen < srcbuf || srcbuf + bytelen < destbuf)
/* No overlapping */
memcpy(destbuf, srcbuf, bytelen);
else
memmove(destbuf, srcbuf, bytelen);
PyBuffer_Release(&srcview);
return 0;
_error:
PyBuffer_Release(&srcview);
return -1;
}
static PyObject *
memory_richcompare(PyObject *v, PyObject *w, int op)
{
Py_buffer vv, ww;
int equal = 0;
PyObject *res;
vv.obj = NULL;
ww.obj = NULL;
if (op != Py_EQ && op != Py_NE)
goto _notimpl;
if (PyObject_GetBuffer(v, &vv, PyBUF_CONTIG_RO) == -1) {
PyErr_Clear();
goto _notimpl;
}
if (PyObject_GetBuffer(w, &ww, PyBUF_CONTIG_RO) == -1) {
PyErr_Clear();
goto _notimpl;
}
if (vv.itemsize != ww.itemsize || vv.len != ww.len)
goto _end;
equal = !memcmp(vv.buf, ww.buf, vv.len);
_end:
PyBuffer_Release(&vv);
PyBuffer_Release(&ww);
if ((equal && op == Py_EQ) || (!equal && op == Py_NE))
res = Py_True;
else
res = Py_False;
Py_INCREF(res);
return res;
_notimpl:
PyBuffer_Release(&vv);
PyBuffer_Release(&ww);
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
static int
memory_traverse(PyMemoryViewObject *self, visitproc visit, void *arg)
{
if (self->base != NULL)
Py_VISIT(self->base);
if (self->view.obj != NULL)
Py_VISIT(self->view.obj);
return 0;
}
static int
memory_clear(PyMemoryViewObject *self)
{
Py_CLEAR(self->base);
PyBuffer_Release(&self->view);
return 0;
}
/* As mapping */
static PyMappingMethods memory_as_mapping = {
(lenfunc)memory_length, /* mp_length */
(binaryfunc)memory_subscript, /* mp_subscript */
(objobjargproc)memory_ass_sub, /* mp_ass_subscript */
};
static PySequenceMethods memory_as_sequence = {
0, /* sq_length */
0, /* sq_concat */
0, /* sq_repeat */
(ssizeargfunc)memory_item, /* sq_item */
};
/* Buffer methods */
static PyBufferProcs memory_as_buffer = {
0, /* bf_getreadbuffer */
0, /* bf_getwritebuffer */
0, /* bf_getsegcount */
0, /* bf_getcharbuffer */
(getbufferproc)memory_getbuf, /* bf_getbuffer */
(releasebufferproc)memory_releasebuf, /* bf_releasebuffer */
};
PyTypeObject PyMemoryView_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"memoryview",
sizeof(PyMemoryViewObject),
0,
(destructor)memory_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)memory_repr, /* tp_repr */
0, /* tp_as_number */
&memory_as_sequence, /* tp_as_sequence */
&memory_as_mapping, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
&memory_as_buffer, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
Py_TPFLAGS_HAVE_NEWBUFFER, /* tp_flags */
memory_doc, /* tp_doc */
(traverseproc)memory_traverse, /* tp_traverse */
(inquiry)memory_clear, /* tp_clear */
memory_richcompare, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
memory_methods, /* tp_methods */
0, /* tp_members */
memory_getsetlist, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
memory_new, /* tp_new */
};

View File

@ -0,0 +1,427 @@
/* Method object implementation */
#include "Python.h"
#include "structmember.h"
/* Free list for method objects to safe malloc/free overhead
* The m_self element is used to chain the objects.
*/
static PyCFunctionObject *free_list = NULL;
static int numfree = 0;
#ifndef PyCFunction_MAXFREELIST
#define PyCFunction_MAXFREELIST 256
#endif
PyObject *
PyCFunction_NewEx(PyMethodDef *ml, PyObject *self, PyObject *module)
{
PyCFunctionObject *op;
op = free_list;
if (op != NULL) {
free_list = (PyCFunctionObject *)(op->m_self);
PyObject_INIT(op, &PyCFunction_Type);
numfree--;
}
else {
op = PyObject_GC_New(PyCFunctionObject, &PyCFunction_Type);
if (op == NULL)
return NULL;
}
op->m_ml = ml;
Py_XINCREF(self);
op->m_self = self;
Py_XINCREF(module);
op->m_module = module;
_PyObject_GC_TRACK(op);
return (PyObject *)op;
}
PyCFunction
PyCFunction_GetFunction(PyObject *op)
{
if (!PyCFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyCFunctionObject *)op) -> m_ml -> ml_meth;
}
PyObject *
PyCFunction_GetSelf(PyObject *op)
{
if (!PyCFunction_Check(op)) {
PyErr_BadInternalCall();
return NULL;
}
return ((PyCFunctionObject *)op) -> m_self;
}
int
PyCFunction_GetFlags(PyObject *op)
{
if (!PyCFunction_Check(op)) {
PyErr_BadInternalCall();
return -1;
}
return ((PyCFunctionObject *)op) -> m_ml -> ml_flags;
}
PyObject *
PyCFunction_Call(PyObject *func, PyObject *arg, PyObject *kw)
{
PyCFunctionObject* f = (PyCFunctionObject*)func;
PyCFunction meth = PyCFunction_GET_FUNCTION(func);
PyObject *self = PyCFunction_GET_SELF(func);
Py_ssize_t size;
switch (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST)) {
case METH_VARARGS:
if (kw == NULL || PyDict_Size(kw) == 0)
return (*meth)(self, arg);
break;
case METH_VARARGS | METH_KEYWORDS:
case METH_OLDARGS | METH_KEYWORDS:
return (*(PyCFunctionWithKeywords)meth)(self, arg, kw);
case METH_NOARGS:
if (kw == NULL || PyDict_Size(kw) == 0) {
size = PyTuple_GET_SIZE(arg);
if (size == 0)
return (*meth)(self, NULL);
PyErr_Format(PyExc_TypeError,
"%.200s() takes no arguments (%zd given)",
f->m_ml->ml_name, size);
return NULL;
}
break;
case METH_O:
if (kw == NULL || PyDict_Size(kw) == 0) {
size = PyTuple_GET_SIZE(arg);
if (size == 1)
return (*meth)(self, PyTuple_GET_ITEM(arg, 0));
PyErr_Format(PyExc_TypeError,
"%.200s() takes exactly one argument (%zd given)",
f->m_ml->ml_name, size);
return NULL;
}
break;
case METH_OLDARGS:
/* the really old style */
if (kw == NULL || PyDict_Size(kw) == 0) {
size = PyTuple_GET_SIZE(arg);
if (size == 1)
arg = PyTuple_GET_ITEM(arg, 0);
else if (size == 0)
arg = NULL;
return (*meth)(self, arg);
}
break;
default:
PyErr_BadInternalCall();
return NULL;
}
PyErr_Format(PyExc_TypeError, "%.200s() takes no keyword arguments",
f->m_ml->ml_name);
return NULL;
}
/* Methods (the standard built-in methods, that is) */
static void
meth_dealloc(PyCFunctionObject *m)
{
_PyObject_GC_UNTRACK(m);
Py_XDECREF(m->m_self);
Py_XDECREF(m->m_module);
if (numfree < PyCFunction_MAXFREELIST) {
m->m_self = (PyObject *)free_list;
free_list = m;
numfree++;
}
else {
PyObject_GC_Del(m);
}
}
static PyObject *
meth_get__doc__(PyCFunctionObject *m, void *closure)
{
const char *doc = m->m_ml->ml_doc;
if (doc != NULL)
return PyString_FromString(doc);
Py_INCREF(Py_None);
return Py_None;
}
static PyObject *
meth_get__name__(PyCFunctionObject *m, void *closure)
{
return PyString_FromString(m->m_ml->ml_name);
}
static int
meth_traverse(PyCFunctionObject *m, visitproc visit, void *arg)
{
Py_VISIT(m->m_self);
Py_VISIT(m->m_module);
return 0;
}
static PyObject *
meth_get__self__(PyCFunctionObject *m, void *closure)
{
PyObject *self;
if (PyEval_GetRestricted()) {
PyErr_SetString(PyExc_RuntimeError,
"method.__self__ not accessible in restricted mode");
return NULL;
}
self = m->m_self;
if (self == NULL)
self = Py_None;
Py_INCREF(self);
return self;
}
static PyGetSetDef meth_getsets [] = {
{"__doc__", (getter)meth_get__doc__, NULL, NULL},
{"__name__", (getter)meth_get__name__, NULL, NULL},
{"__self__", (getter)meth_get__self__, NULL, NULL},
{0}
};
#define OFF(x) offsetof(PyCFunctionObject, x)
static PyMemberDef meth_members[] = {
{"__module__", T_OBJECT, OFF(m_module), PY_WRITE_RESTRICTED},
{NULL}
};
static PyObject *
meth_repr(PyCFunctionObject *m)
{
if (m->m_self == NULL)
return PyString_FromFormat("<built-in function %s>",
m->m_ml->ml_name);
return PyString_FromFormat("<built-in method %s of %s object at %p>",
m->m_ml->ml_name,
m->m_self->ob_type->tp_name,
m->m_self);
}
static int
meth_compare(PyCFunctionObject *a, PyCFunctionObject *b)
{
if (a->m_self != b->m_self)
return (a->m_self < b->m_self) ? -1 : 1;
if (a->m_ml->ml_meth == b->m_ml->ml_meth)
return 0;
if (strcmp(a->m_ml->ml_name, b->m_ml->ml_name) < 0)
return -1;
else
return 1;
}
static PyObject *
meth_richcompare(PyObject *self, PyObject *other, int op)
{
PyCFunctionObject *a, *b;
PyObject *res;
int eq;
if (op != Py_EQ && op != Py_NE) {
/* Py3K warning if comparison isn't == or !=. */
if (PyErr_WarnPy3k("builtin_function_or_method order "
"comparisons not supported in 3.x", 1) < 0) {
return NULL;
}
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
else if (!PyCFunction_Check(self) || !PyCFunction_Check(other)) {
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
a = (PyCFunctionObject *)self;
b = (PyCFunctionObject *)other;
eq = a->m_self == b->m_self;
if (eq)
eq = a->m_ml->ml_meth == b->m_ml->ml_meth;
if (op == Py_EQ)
res = eq ? Py_True : Py_False;
else
res = eq ? Py_False : Py_True;
Py_INCREF(res);
return res;
}
static long
meth_hash(PyCFunctionObject *a)
{
long x,y;
if (a->m_self == NULL)
x = 0;
else {
x = PyObject_Hash(a->m_self);
if (x == -1)
return -1;
}
y = _Py_HashPointer((void*)(a->m_ml->ml_meth));
if (y == -1)
return -1;
x ^= y;
if (x == -1)
x = -2;
return x;
}
PyTypeObject PyCFunction_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"builtin_function_or_method",
sizeof(PyCFunctionObject),
0,
(destructor)meth_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
(cmpfunc)meth_compare, /* tp_compare */
(reprfunc)meth_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
(hashfunc)meth_hash, /* tp_hash */
PyCFunction_Call, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
0, /* tp_doc */
(traverseproc)meth_traverse, /* tp_traverse */
0, /* tp_clear */
meth_richcompare, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
meth_members, /* tp_members */
meth_getsets, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
};
/* List all methods in a chain -- helper for findmethodinchain */
static PyObject *
listmethodchain(PyMethodChain *chain)
{
PyMethodChain *c;
PyMethodDef *ml;
int i, n;
PyObject *v;
n = 0;
for (c = chain; c != NULL; c = c->link) {
for (ml = c->methods; ml->ml_name != NULL; ml++)
n++;
}
v = PyList_New(n);
if (v == NULL)
return NULL;
i = 0;
for (c = chain; c != NULL; c = c->link) {
for (ml = c->methods; ml->ml_name != NULL; ml++) {
PyList_SetItem(v, i, PyString_FromString(ml->ml_name));
i++;
}
}
if (PyErr_Occurred()) {
Py_DECREF(v);
return NULL;
}
PyList_Sort(v);
return v;
}
/* Find a method in a method chain */
PyObject *
Py_FindMethodInChain(PyMethodChain *chain, PyObject *self, const char *name)
{
if (name[0] == '_' && name[1] == '_') {
if (strcmp(name, "__methods__") == 0) {
if (PyErr_WarnPy3k("__methods__ not supported in 3.x",
1) < 0)
return NULL;
return listmethodchain(chain);
}
if (strcmp(name, "__doc__") == 0) {
const char *doc = self->ob_type->tp_doc;
if (doc != NULL)
return PyString_FromString(doc);
}
}
while (chain != NULL) {
PyMethodDef *ml = chain->methods;
for (; ml->ml_name != NULL; ml++) {
if (name[0] == ml->ml_name[0] &&
strcmp(name+1, ml->ml_name+1) == 0)
/* XXX */
return PyCFunction_New(ml, self);
}
chain = chain->link;
}
PyErr_SetString(PyExc_AttributeError, name);
return NULL;
}
/* Find a method in a single method list */
PyObject *
Py_FindMethod(PyMethodDef *methods, PyObject *self, const char *name)
{
PyMethodChain chain;
chain.methods = methods;
chain.link = NULL;
return Py_FindMethodInChain(&chain, self, name);
}
/* Clear out the free list */
int
PyCFunction_ClearFreeList(void)
{
int freelist_size = numfree;
while (free_list) {
PyCFunctionObject *v = free_list;
free_list = (PyCFunctionObject *)(v->m_self);
PyObject_GC_Del(v);
numfree--;
}
assert(numfree == 0);
return freelist_size;
}
void
PyCFunction_Fini(void)
{
(void)PyCFunction_ClearFreeList();
}
/* PyCFunction_New() is now just a macro that calls PyCFunction_NewEx(),
but it's part of the API so we need to keep a function around that
existing C extensions can call.
*/
#undef PyCFunction_New
PyAPI_FUNC(PyObject *) PyCFunction_New(PyMethodDef *, PyObject *);
PyObject *
PyCFunction_New(PyMethodDef *ml, PyObject *self)
{
return PyCFunction_NewEx(ml, self, NULL);
}

View File

@ -0,0 +1,262 @@
/* Module object implementation */
#include "Python.h"
#include "structmember.h"
typedef struct {
PyObject_HEAD
PyObject *md_dict;
} PyModuleObject;
static PyMemberDef module_members[] = {
{"__dict__", T_OBJECT, offsetof(PyModuleObject, md_dict), READONLY},
{0}
};
PyObject *
PyModule_New(const char *name)
{
PyModuleObject *m;
PyObject *nameobj;
m = PyObject_GC_New(PyModuleObject, &PyModule_Type);
if (m == NULL)
return NULL;
nameobj = PyString_FromString(name);
m->md_dict = PyDict_New();
if (m->md_dict == NULL || nameobj == NULL)
goto fail;
if (PyDict_SetItemString(m->md_dict, "__name__", nameobj) != 0)
goto fail;
if (PyDict_SetItemString(m->md_dict, "__doc__", Py_None) != 0)
goto fail;
if (PyDict_SetItemString(m->md_dict, "__package__", Py_None) != 0)
goto fail;
Py_DECREF(nameobj);
PyObject_GC_Track(m);
return (PyObject *)m;
fail:
Py_XDECREF(nameobj);
Py_DECREF(m);
return NULL;
}
PyObject *
PyModule_GetDict(PyObject *m)
{
PyObject *d;
if (!PyModule_Check(m)) {
PyErr_BadInternalCall();
return NULL;
}
d = ((PyModuleObject *)m) -> md_dict;
if (d == NULL)
((PyModuleObject *)m) -> md_dict = d = PyDict_New();
return d;
}
char *
PyModule_GetName(PyObject *m)
{
PyObject *d;
PyObject *nameobj;
if (!PyModule_Check(m)) {
PyErr_BadArgument();
return NULL;
}
d = ((PyModuleObject *)m)->md_dict;
if (d == NULL ||
(nameobj = PyDict_GetItemString(d, "__name__")) == NULL ||
!PyString_Check(nameobj))
{
PyErr_SetString(PyExc_SystemError, "nameless module");
return NULL;
}
return PyString_AsString(nameobj);
}
char *
PyModule_GetFilename(PyObject *m)
{
PyObject *d;
PyObject *fileobj;
if (!PyModule_Check(m)) {
PyErr_BadArgument();
return NULL;
}
d = ((PyModuleObject *)m)->md_dict;
if (d == NULL ||
(fileobj = PyDict_GetItemString(d, "__file__")) == NULL ||
!PyString_Check(fileobj))
{
PyErr_SetString(PyExc_SystemError, "module filename missing");
return NULL;
}
return PyString_AsString(fileobj);
}
void
_PyModule_Clear(PyObject *m)
{
/* To make the execution order of destructors for global
objects a bit more predictable, we first zap all objects
whose name starts with a single underscore, before we clear
the entire dictionary. We zap them by replacing them with
None, rather than deleting them from the dictionary, to
avoid rehashing the dictionary (to some extent). */
Py_ssize_t pos;
PyObject *key, *value;
PyObject *d;
d = ((PyModuleObject *)m)->md_dict;
if (d == NULL)
return;
/* First, clear only names starting with a single underscore */
pos = 0;
while (PyDict_Next(d, &pos, &key, &value)) {
if (value != Py_None && PyString_Check(key)) {
char *s = PyString_AsString(key);
if (s[0] == '_' && s[1] != '_') {
if (Py_VerboseFlag > 1)
PySys_WriteStderr("# clear[1] %s\n", s);
if (PyDict_SetItem(d, key, Py_None) != 0)
PyErr_Clear();
}
}
}
/* Next, clear all names except for __builtins__ */
pos = 0;
while (PyDict_Next(d, &pos, &key, &value)) {
if (value != Py_None && PyString_Check(key)) {
char *s = PyString_AsString(key);
if (s[0] != '_' || strcmp(s, "__builtins__") != 0) {
if (Py_VerboseFlag > 1)
PySys_WriteStderr("# clear[2] %s\n", s);
if (PyDict_SetItem(d, key, Py_None) != 0)
PyErr_Clear();
}
}
}
/* Note: we leave __builtins__ in place, so that destructors
of non-global objects defined in this module can still use
builtins, in particularly 'None'. */
}
/* Methods */
static int
module_init(PyModuleObject *m, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {"name", "doc", NULL};
PyObject *dict, *name = Py_None, *doc = Py_None;
if (!PyArg_ParseTupleAndKeywords(args, kwds, "S|O:module.__init__",
kwlist, &name, &doc))
return -1;
dict = m->md_dict;
if (dict == NULL) {
dict = PyDict_New();
if (dict == NULL)
return -1;
m->md_dict = dict;
}
if (PyDict_SetItemString(dict, "__name__", name) < 0)
return -1;
if (PyDict_SetItemString(dict, "__doc__", doc) < 0)
return -1;
return 0;
}
static void
module_dealloc(PyModuleObject *m)
{
PyObject_GC_UnTrack(m);
if (m->md_dict != NULL) {
_PyModule_Clear((PyObject *)m);
Py_DECREF(m->md_dict);
}
Py_TYPE(m)->tp_free((PyObject *)m);
}
static PyObject *
module_repr(PyModuleObject *m)
{
char *name;
char *filename;
name = PyModule_GetName((PyObject *)m);
if (name == NULL) {
PyErr_Clear();
name = "?";
}
filename = PyModule_GetFilename((PyObject *)m);
if (filename == NULL) {
PyErr_Clear();
return PyString_FromFormat("<module '%s' (built-in)>", name);
}
return PyString_FromFormat("<module '%s' from '%s'>", name, filename);
}
/* We only need a traverse function, no clear function: If the module
is in a cycle, md_dict will be cleared as well, which will break
the cycle. */
static int
module_traverse(PyModuleObject *m, visitproc visit, void *arg)
{
Py_VISIT(m->md_dict);
return 0;
}
PyDoc_STRVAR(module_doc,
"module(name[, doc])\n\
\n\
Create a module object.\n\
The name must be a string; the optional doc argument can have any type.");
PyTypeObject PyModule_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"module", /* tp_name */
sizeof(PyModuleObject), /* tp_size */
0, /* tp_itemsize */
(destructor)module_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)module_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
PyObject_GenericSetAttr, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
Py_TPFLAGS_BASETYPE, /* tp_flags */
module_doc, /* tp_doc */
(traverseproc)module_traverse, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
module_members, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
offsetof(PyModuleObject, md_dict), /* tp_dictoffset */
(initproc)module_init, /* tp_init */
PyType_GenericAlloc, /* tp_alloc */
PyType_GenericNew, /* tp_new */
PyObject_GC_Del, /* tp_free */
};

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,347 @@
/* Range object implementation */
#include "Python.h"
typedef struct {
PyObject_HEAD
long start;
long step;
long len;
} rangeobject;
/* Return number of items in range (lo, hi, step). step != 0
* required. The result always fits in an unsigned long.
*/
static unsigned long
get_len_of_range(long lo, long hi, long step)
{
/* -------------------------------------------------------------
If step > 0 and lo >= hi, or step < 0 and lo <= hi, the range is empty.
Else for step > 0, if n values are in the range, the last one is
lo + (n-1)*step, which must be <= hi-1. Rearranging,
n <= (hi - lo - 1)/step + 1, so taking the floor of the RHS gives
the proper value. Since lo < hi in this case, hi-lo-1 >= 0, so
the RHS is non-negative and so truncation is the same as the
floor. Letting M be the largest positive long, the worst case
for the RHS numerator is hi=M, lo=-M-1, and then
hi-lo-1 = M-(-M-1)-1 = 2*M. Therefore unsigned long has enough
precision to compute the RHS exactly. The analysis for step < 0
is similar.
---------------------------------------------------------------*/
assert(step != 0);
if (step > 0 && lo < hi)
return 1UL + (hi - 1UL - lo) / step;
else if (step < 0 && lo > hi)
return 1UL + (lo - 1UL - hi) / (0UL - step);
else
return 0UL;
}
/* Return a stop value suitable for reconstructing the xrange from
* a (start, stop, step) triple. Used in range_repr and range_reduce.
* Computes start + len * step, clipped to the range [LONG_MIN, LONG_MAX].
*/
static long
get_stop_for_range(rangeobject *r)
{
long last;
if (r->len == 0)
return r->start;
/* The tricky bit is avoiding overflow. We first compute the last entry in
the xrange, start + (len - 1) * step, which is guaranteed to lie within
the range of a long, and then add step to it. See the range_reverse
comments for an explanation of the casts below.
*/
last = (long)(r->start + (unsigned long)(r->len - 1) * r->step);
if (r->step > 0)
return last > LONG_MAX - r->step ? LONG_MAX : last + r->step;
else
return last < LONG_MIN - r->step ? LONG_MIN : last + r->step;
}
static PyObject *
range_new(PyTypeObject *type, PyObject *args, PyObject *kw)
{
rangeobject *obj;
long ilow = 0, ihigh = 0, istep = 1;
unsigned long n;
if (!_PyArg_NoKeywords("xrange()", kw))
return NULL;
if (PyTuple_Size(args) <= 1) {
if (!PyArg_ParseTuple(args,
"l;xrange() requires 1-3 int arguments",
&ihigh))
return NULL;
}
else {
if (!PyArg_ParseTuple(args,
"ll|l;xrange() requires 1-3 int arguments",
&ilow, &ihigh, &istep))
return NULL;
}
if (istep == 0) {
PyErr_SetString(PyExc_ValueError, "xrange() arg 3 must not be zero");
return NULL;
}
n = get_len_of_range(ilow, ihigh, istep);
if (n > (unsigned long)LONG_MAX || (long)n > PY_SSIZE_T_MAX) {
PyErr_SetString(PyExc_OverflowError,
"xrange() result has too many items");
return NULL;
}
obj = PyObject_New(rangeobject, &PyRange_Type);
if (obj == NULL)
return NULL;
obj->start = ilow;
obj->len = (long)n;
obj->step = istep;
return (PyObject *) obj;
}
PyDoc_STRVAR(range_doc,
"xrange(stop) -> xrange object\n\
xrange(start, stop[, step]) -> xrange object\n\
\n\
Like range(), but instead of returning a list, returns an object that\n\
generates the numbers in the range on demand. For looping, this is \n\
slightly faster than range() and more memory efficient.");
static PyObject *
range_item(rangeobject *r, Py_ssize_t i)
{
if (i < 0 || i >= r->len) {
PyErr_SetString(PyExc_IndexError,
"xrange object index out of range");
return NULL;
}
/* do calculation entirely using unsigned longs, to avoid
undefined behaviour due to signed overflow. */
return PyInt_FromLong((long)(r->start + (unsigned long)i * r->step));
}
static Py_ssize_t
range_length(rangeobject *r)
{
return (Py_ssize_t)(r->len);
}
static PyObject *
range_repr(rangeobject *r)
{
PyObject *rtn;
if (r->start == 0 && r->step == 1)
rtn = PyString_FromFormat("xrange(%ld)",
get_stop_for_range(r));
else if (r->step == 1)
rtn = PyString_FromFormat("xrange(%ld, %ld)",
r->start,
get_stop_for_range(r));
else
rtn = PyString_FromFormat("xrange(%ld, %ld, %ld)",
r->start,
get_stop_for_range(r),
r->step);
return rtn;
}
/* Pickling support */
static PyObject *
range_reduce(rangeobject *r, PyObject *args)
{
return Py_BuildValue("(O(lll))", Py_TYPE(r),
r->start,
get_stop_for_range(r),
r->step);
}
static PySequenceMethods range_as_sequence = {
(lenfunc)range_length, /* sq_length */
0, /* sq_concat */
0, /* sq_repeat */
(ssizeargfunc)range_item, /* sq_item */
0, /* sq_slice */
};
static PyObject * range_iter(PyObject *seq);
static PyObject * range_reverse(PyObject *seq);
PyDoc_STRVAR(reverse_doc,
"Returns a reverse iterator.");
static PyMethodDef range_methods[] = {
{"__reversed__", (PyCFunction)range_reverse, METH_NOARGS, reverse_doc},
{"__reduce__", (PyCFunction)range_reduce, METH_VARARGS},
{NULL, NULL} /* sentinel */
};
PyTypeObject PyRange_Type = {
PyObject_HEAD_INIT(&PyType_Type)
0, /* Number of items for varobject */
"xrange", /* Name of this type */
sizeof(rangeobject), /* Basic object size */
0, /* Item size for varobject */
(destructor)PyObject_Del, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)range_repr, /* tp_repr */
0, /* tp_as_number */
&range_as_sequence, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
range_doc, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
range_iter, /* tp_iter */
0, /* tp_iternext */
range_methods, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
range_new, /* tp_new */
};
/*********************** Xrange Iterator **************************/
typedef struct {
PyObject_HEAD
long index;
long start;
long step;
long len;
} rangeiterobject;
static PyObject *
rangeiter_next(rangeiterobject *r)
{
if (r->index < r->len)
return PyInt_FromLong(r->start + (r->index++) * r->step);
return NULL;
}
static PyObject *
rangeiter_len(rangeiterobject *r)
{
return PyInt_FromLong(r->len - r->index);
}
PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list(it)).");
static PyMethodDef rangeiter_methods[] = {
{"__length_hint__", (PyCFunction)rangeiter_len, METH_NOARGS, length_hint_doc},
{NULL, NULL} /* sentinel */
};
static PyTypeObject Pyrangeiter_Type = {
PyObject_HEAD_INIT(&PyType_Type)
0, /* ob_size */
"rangeiterator", /* tp_name */
sizeof(rangeiterobject), /* tp_basicsize */
0, /* tp_itemsize */
/* methods */
(destructor)PyObject_Del, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
0, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
PyObject_SelfIter, /* tp_iter */
(iternextfunc)rangeiter_next, /* tp_iternext */
rangeiter_methods, /* tp_methods */
0,
};
static PyObject *
range_iter(PyObject *seq)
{
rangeiterobject *it;
if (!PyRange_Check(seq)) {
PyErr_BadInternalCall();
return NULL;
}
it = PyObject_New(rangeiterobject, &Pyrangeiter_Type);
if (it == NULL)
return NULL;
it->index = 0;
it->start = ((rangeobject *)seq)->start;
it->step = ((rangeobject *)seq)->step;
it->len = ((rangeobject *)seq)->len;
return (PyObject *)it;
}
static PyObject *
range_reverse(PyObject *seq)
{
rangeiterobject *it;
long start, step, len;
if (!PyRange_Check(seq)) {
PyErr_BadInternalCall();
return NULL;
}
it = PyObject_New(rangeiterobject, &Pyrangeiter_Type);
if (it == NULL)
return NULL;
start = ((rangeobject *)seq)->start;
step = ((rangeobject *)seq)->step;
len = ((rangeobject *)seq)->len;
it->index = 0;
it->len = len;
/* the casts below guard against signed overflow by turning it
into unsigned overflow instead. The correctness of this
code still depends on conversion from unsigned long to long
wrapping modulo ULONG_MAX+1, which isn't guaranteed (see
C99 6.3.1.3p3) but seems to hold in practice for all
platforms we're likely to meet.
If step == LONG_MIN then we still end up with LONG_MIN
after negation; but this works out, since we've still got
the correct value modulo ULONG_MAX+1, and the range_item
calculation is also done modulo ULONG_MAX+1.
*/
it->start = (long)(start + (unsigned long)(len-1) * step);
it->step = (long)(0UL-step);
return (PyObject *)it;
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,362 @@
/*
Written by Jim Hugunin and Chris Chase.
This includes both the singular ellipsis object and slice objects.
Guido, feel free to do whatever you want in the way of copyrights
for this file.
*/
/*
Py_Ellipsis encodes the '...' rubber index token. It is similar to
the Py_NoneStruct in that there is no way to create other objects of
this type and there is exactly one in existence.
*/
#include "Python.h"
#include "structmember.h"
static PyObject *
ellipsis_repr(PyObject *op)
{
return PyString_FromString("Ellipsis");
}
PyTypeObject PyEllipsis_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"ellipsis", /* tp_name */
0, /* tp_basicsize */
0, /* tp_itemsize */
0, /*never called*/ /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
ellipsis_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
};
PyObject _Py_EllipsisObject = {
_PyObject_EXTRA_INIT
1, &PyEllipsis_Type
};
/* Slice object implementation
start, stop, and step are python objects with None indicating no
index is present.
*/
PyObject *
PySlice_New(PyObject *start, PyObject *stop, PyObject *step)
{
PySliceObject *obj = PyObject_New(PySliceObject, &PySlice_Type);
if (obj == NULL)
return NULL;
if (step == NULL) step = Py_None;
Py_INCREF(step);
if (start == NULL) start = Py_None;
Py_INCREF(start);
if (stop == NULL) stop = Py_None;
Py_INCREF(stop);
obj->step = step;
obj->start = start;
obj->stop = stop;
return (PyObject *) obj;
}
PyObject *
_PySlice_FromIndices(Py_ssize_t istart, Py_ssize_t istop)
{
PyObject *start, *end, *slice;
start = PyInt_FromSsize_t(istart);
if (!start)
return NULL;
end = PyInt_FromSsize_t(istop);
if (!end) {
Py_DECREF(start);
return NULL;
}
slice = PySlice_New(start, end, NULL);
Py_DECREF(start);
Py_DECREF(end);
return slice;
}
int
PySlice_GetIndices(PySliceObject *r, Py_ssize_t length,
Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step)
{
/* XXX support long ints */
if (r->step == Py_None) {
*step = 1;
} else {
if (!PyInt_Check(r->step) && !PyLong_Check(r->step)) return -1;
*step = PyInt_AsSsize_t(r->step);
}
if (r->start == Py_None) {
*start = *step < 0 ? length-1 : 0;
} else {
if (!PyInt_Check(r->start) && !PyLong_Check(r->step)) return -1;
*start = PyInt_AsSsize_t(r->start);
if (*start < 0) *start += length;
}
if (r->stop == Py_None) {
*stop = *step < 0 ? -1 : length;
} else {
if (!PyInt_Check(r->stop) && !PyLong_Check(r->step)) return -1;
*stop = PyInt_AsSsize_t(r->stop);
if (*stop < 0) *stop += length;
}
if (*stop > length) return -1;
if (*start >= length) return -1;
if (*step == 0) return -1;
return 0;
}
int
PySlice_GetIndicesEx(PySliceObject *r, Py_ssize_t length,
Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step, Py_ssize_t *slicelength)
{
/* this is harder to get right than you might think */
Py_ssize_t defstart, defstop;
if (r->step == Py_None) {
*step = 1;
}
else {
if (!_PyEval_SliceIndex(r->step, step)) return -1;
if (*step == 0) {
PyErr_SetString(PyExc_ValueError,
"slice step cannot be zero");
return -1;
}
}
defstart = *step < 0 ? length-1 : 0;
defstop = *step < 0 ? -1 : length;
if (r->start == Py_None) {
*start = defstart;
}
else {
if (!_PyEval_SliceIndex(r->start, start)) return -1;
if (*start < 0) *start += length;
if (*start < 0) *start = (*step < 0) ? -1 : 0;
if (*start >= length)
*start = (*step < 0) ? length - 1 : length;
}
if (r->stop == Py_None) {
*stop = defstop;
}
else {
if (!_PyEval_SliceIndex(r->stop, stop)) return -1;
if (*stop < 0) *stop += length;
if (*stop < 0) *stop = (*step < 0) ? -1 : 0;
if (*stop >= length)
*stop = (*step < 0) ? length - 1 : length;
}
if ((*step < 0 && *stop >= *start)
|| (*step > 0 && *start >= *stop)) {
*slicelength = 0;
}
else if (*step < 0) {
*slicelength = (*stop-*start+1)/(*step)+1;
}
else {
*slicelength = (*stop-*start-1)/(*step)+1;
}
return 0;
}
static PyObject *
slice_new(PyTypeObject *type, PyObject *args, PyObject *kw)
{
PyObject *start, *stop, *step;
start = stop = step = NULL;
if (!_PyArg_NoKeywords("slice()", kw))
return NULL;
if (!PyArg_UnpackTuple(args, "slice", 1, 3, &start, &stop, &step))
return NULL;
/* This swapping of stop and start is to maintain similarity with
range(). */
if (stop == NULL) {
stop = start;
start = NULL;
}
return PySlice_New(start, stop, step);
}
PyDoc_STRVAR(slice_doc,
"slice(stop)\n\
slice(start, stop[, step])\n\
\n\
Create a slice object. This is used for extended slicing (e.g. a[0:10:2]).");
static void
slice_dealloc(PySliceObject *r)
{
Py_DECREF(r->step);
Py_DECREF(r->start);
Py_DECREF(r->stop);
PyObject_Del(r);
}
static PyObject *
slice_repr(PySliceObject *r)
{
PyObject *s, *comma;
s = PyString_FromString("slice(");
comma = PyString_FromString(", ");
PyString_ConcatAndDel(&s, PyObject_Repr(r->start));
PyString_Concat(&s, comma);
PyString_ConcatAndDel(&s, PyObject_Repr(r->stop));
PyString_Concat(&s, comma);
PyString_ConcatAndDel(&s, PyObject_Repr(r->step));
PyString_ConcatAndDel(&s, PyString_FromString(")"));
Py_DECREF(comma);
return s;
}
static PyMemberDef slice_members[] = {
{"start", T_OBJECT, offsetof(PySliceObject, start), READONLY},
{"stop", T_OBJECT, offsetof(PySliceObject, stop), READONLY},
{"step", T_OBJECT, offsetof(PySliceObject, step), READONLY},
{0}
};
static PyObject*
slice_indices(PySliceObject* self, PyObject* len)
{
Py_ssize_t ilen, start, stop, step, slicelength;
ilen = PyNumber_AsSsize_t(len, PyExc_OverflowError);
if (ilen == -1 && PyErr_Occurred()) {
return NULL;
}
if (PySlice_GetIndicesEx(self, ilen, &start, &stop,
&step, &slicelength) < 0) {
return NULL;
}
return Py_BuildValue("(nnn)", start, stop, step);
}
PyDoc_STRVAR(slice_indices_doc,
"S.indices(len) -> (start, stop, stride)\n\
\n\
Assuming a sequence of length len, calculate the start and stop\n\
indices, and the stride length of the extended slice described by\n\
S. Out of bounds indices are clipped in a manner consistent with the\n\
handling of normal slices.");
static PyObject *
slice_reduce(PySliceObject* self)
{
return Py_BuildValue("O(OOO)", Py_TYPE(self), self->start, self->stop, self->step);
}
PyDoc_STRVAR(reduce_doc, "Return state information for pickling.");
static PyMethodDef slice_methods[] = {
{"indices", (PyCFunction)slice_indices,
METH_O, slice_indices_doc},
{"__reduce__", (PyCFunction)slice_reduce,
METH_NOARGS, reduce_doc},
{NULL, NULL}
};
static int
slice_compare(PySliceObject *v, PySliceObject *w)
{
int result = 0;
if (v == w)
return 0;
if (PyObject_Cmp(v->start, w->start, &result) < 0)
return -2;
if (result != 0)
return result;
if (PyObject_Cmp(v->stop, w->stop, &result) < 0)
return -2;
if (result != 0)
return result;
if (PyObject_Cmp(v->step, w->step, &result) < 0)
return -2;
return result;
}
static long
slice_hash(PySliceObject *v)
{
PyErr_SetString(PyExc_TypeError, "unhashable type");
return -1L;
}
PyTypeObject PySlice_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"slice", /* Name of this type */
sizeof(PySliceObject), /* Basic object size */
0, /* Item size for varobject */
(destructor)slice_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
(cmpfunc)slice_compare, /* tp_compare */
(reprfunc)slice_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
(hashfunc)slice_hash, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
slice_doc, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
slice_methods, /* tp_methods */
slice_members, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
slice_new, /* tp_new */
};

View File

@ -0,0 +1,40 @@
bits shared by the stringobject and unicodeobject implementations (and
possibly other modules, in a not too distant future).
the stuff in here is included into relevant places; see the individual
source files for details.
--------------------------------------------------------------------
the following defines used by the different modules:
STRINGLIB_CHAR
the type used to hold a character (char or Py_UNICODE)
STRINGLIB_EMPTY
a PyObject representing the empty string, only to be used if
STRINGLIB_MUTABLE is 0
Py_ssize_t STRINGLIB_LEN(PyObject*)
returns the length of the given string object (which must be of the
right type)
PyObject* STRINGLIB_NEW(STRINGLIB_CHAR*, Py_ssize_t)
creates a new string object
STRINGLIB_CHAR* STRINGLIB_STR(PyObject*)
returns the pointer to the character data for the given string
object (which must be of the right type)
int STRINGLIB_CHECK_EXACT(PyObject *)
returns true if the object is an instance of our type, not a subclass
STRINGLIB_MUTABLE
must be 0 or 1 to tell the cpp macros in stringlib code if the object
being operated on is mutable or not

View File

@ -0,0 +1,30 @@
/* stringlib: count implementation */
#ifndef STRINGLIB_COUNT_H
#define STRINGLIB_COUNT_H
#ifndef STRINGLIB_FASTSEARCH_H
#error must include "stringlib/fastsearch.h" before including this module
#endif
Py_LOCAL_INLINE(Py_ssize_t)
stringlib_count(const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sub, Py_ssize_t sub_len,
Py_ssize_t maxcount)
{
Py_ssize_t count;
if (str_len < 0)
return 0; /* start > len(str) */
if (sub_len == 0)
return (str_len < maxcount) ? str_len + 1 : maxcount;
count = fastsearch(str, str_len, sub, sub_len, maxcount, FAST_COUNT);
if (count < 0)
return 0; /* no match */
return count;
}
#endif

View File

@ -0,0 +1,109 @@
/* NOTE: this API is -ONLY- for use with single byte character strings. */
/* Do not use it with Unicode. */
#include "bytes_methods.h"
static PyObject*
stringlib_isspace(PyObject *self)
{
return _Py_bytes_isspace(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
static PyObject*
stringlib_isalpha(PyObject *self)
{
return _Py_bytes_isalpha(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
static PyObject*
stringlib_isalnum(PyObject *self)
{
return _Py_bytes_isalnum(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
static PyObject*
stringlib_isdigit(PyObject *self)
{
return _Py_bytes_isdigit(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
static PyObject*
stringlib_islower(PyObject *self)
{
return _Py_bytes_islower(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
static PyObject*
stringlib_isupper(PyObject *self)
{
return _Py_bytes_isupper(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
static PyObject*
stringlib_istitle(PyObject *self)
{
return _Py_bytes_istitle(STRINGLIB_STR(self), STRINGLIB_LEN(self));
}
/* functions that return a new object partially translated by ctype funcs: */
static PyObject*
stringlib_lower(PyObject *self)
{
PyObject* newobj;
newobj = STRINGLIB_NEW(NULL, STRINGLIB_LEN(self));
if (!newobj)
return NULL;
_Py_bytes_lower(STRINGLIB_STR(newobj), STRINGLIB_STR(self),
STRINGLIB_LEN(self));
return newobj;
}
static PyObject*
stringlib_upper(PyObject *self)
{
PyObject* newobj;
newobj = STRINGLIB_NEW(NULL, STRINGLIB_LEN(self));
if (!newobj)
return NULL;
_Py_bytes_upper(STRINGLIB_STR(newobj), STRINGLIB_STR(self),
STRINGLIB_LEN(self));
return newobj;
}
static PyObject*
stringlib_title(PyObject *self)
{
PyObject* newobj;
newobj = STRINGLIB_NEW(NULL, STRINGLIB_LEN(self));
if (!newobj)
return NULL;
_Py_bytes_title(STRINGLIB_STR(newobj), STRINGLIB_STR(self),
STRINGLIB_LEN(self));
return newobj;
}
static PyObject*
stringlib_capitalize(PyObject *self)
{
PyObject* newobj;
newobj = STRINGLIB_NEW(NULL, STRINGLIB_LEN(self));
if (!newobj)
return NULL;
_Py_bytes_capitalize(STRINGLIB_STR(newobj), STRINGLIB_STR(self),
STRINGLIB_LEN(self));
return newobj;
}
static PyObject*
stringlib_swapcase(PyObject *self)
{
PyObject* newobj;
newobj = STRINGLIB_NEW(NULL, STRINGLIB_LEN(self));
if (!newobj)
return NULL;
_Py_bytes_swapcase(STRINGLIB_STR(newobj), STRINGLIB_STR(self),
STRINGLIB_LEN(self));
return newobj;
}

View File

@ -0,0 +1,160 @@
/* stringlib: fastsearch implementation */
#ifndef STRINGLIB_FASTSEARCH_H
#define STRINGLIB_FASTSEARCH_H
/* fast search/count implementation, based on a mix between boyer-
moore and horspool, with a few more bells and whistles on the top.
for some more background, see: http://effbot.org/zone/stringlib.htm */
/* note: fastsearch may access s[n], which isn't a problem when using
Python's ordinary string types, but may cause problems if you're
using this code in other contexts. also, the count mode returns -1
if there cannot possible be a match in the target string, and 0 if
it has actually checked for matches, but didn't find any. callers
beware! */
#define FAST_COUNT 0
#define FAST_SEARCH 1
#define FAST_RSEARCH 2
#if LONG_BIT >= 128
#define STRINGLIB_BLOOM_WIDTH 128
#elif LONG_BIT >= 64
#define STRINGLIB_BLOOM_WIDTH 64
#elif LONG_BIT >= 32
#define STRINGLIB_BLOOM_WIDTH 32
#else
#error "LONG_BIT is smaller than 32"
#endif
#define STRINGLIB_BLOOM_ADD(mask, ch) \
((mask |= (1UL << ((ch) & (STRINGLIB_BLOOM_WIDTH -1)))))
#define STRINGLIB_BLOOM(mask, ch) \
((mask & (1UL << ((ch) & (STRINGLIB_BLOOM_WIDTH -1)))))
Py_LOCAL_INLINE(Py_ssize_t)
fastsearch(const STRINGLIB_CHAR* s, Py_ssize_t n,
const STRINGLIB_CHAR* p, Py_ssize_t m,
Py_ssize_t maxcount, int mode)
{
unsigned long mask;
Py_ssize_t skip, count = 0;
Py_ssize_t i, j, mlast, w;
w = n - m;
if (w < 0 || (mode == FAST_COUNT && maxcount == 0))
return -1;
/* look for special cases */
if (m <= 1) {
if (m <= 0)
return -1;
/* use special case for 1-character strings */
if (mode == FAST_COUNT) {
for (i = 0; i < n; i++)
if (s[i] == p[0]) {
count++;
if (count == maxcount)
return maxcount;
}
return count;
} else if (mode == FAST_SEARCH) {
for (i = 0; i < n; i++)
if (s[i] == p[0])
return i;
} else { /* FAST_RSEARCH */
for (i = n - 1; i > -1; i--)
if (s[i] == p[0])
return i;
}
return -1;
}
mlast = m - 1;
skip = mlast - 1;
mask = 0;
if (mode != FAST_RSEARCH) {
/* create compressed boyer-moore delta 1 table */
/* process pattern[:-1] */
for (i = 0; i < mlast; i++) {
STRINGLIB_BLOOM_ADD(mask, p[i]);
if (p[i] == p[mlast])
skip = mlast - i - 1;
}
/* process pattern[-1] outside the loop */
STRINGLIB_BLOOM_ADD(mask, p[mlast]);
for (i = 0; i <= w; i++) {
/* note: using mlast in the skip path slows things down on x86 */
if (s[i+m-1] == p[m-1]) {
/* candidate match */
for (j = 0; j < mlast; j++)
if (s[i+j] != p[j])
break;
if (j == mlast) {
/* got a match! */
if (mode != FAST_COUNT)
return i;
count++;
if (count == maxcount)
return maxcount;
i = i + mlast;
continue;
}
/* miss: check if next character is part of pattern */
if (!STRINGLIB_BLOOM(mask, s[i+m]))
i = i + m;
else
i = i + skip;
} else {
/* skip: check if next character is part of pattern */
if (!STRINGLIB_BLOOM(mask, s[i+m]))
i = i + m;
}
}
} else { /* FAST_RSEARCH */
/* create compressed boyer-moore delta 1 table */
/* process pattern[0] outside the loop */
STRINGLIB_BLOOM_ADD(mask, p[0]);
/* process pattern[:0:-1] */
for (i = mlast; i > 0; i--) {
STRINGLIB_BLOOM_ADD(mask, p[i]);
if (p[i] == p[0])
skip = i - 1;
}
for (i = w; i >= 0; i--) {
if (s[i] == p[0]) {
/* candidate match */
for (j = mlast; j > 0; j--)
if (s[i+j] != p[j])
break;
if (j == 0)
/* got a match! */
return i;
/* miss: check if previous character is part of pattern */
if (i > 0 && !STRINGLIB_BLOOM(mask, s[i-1]))
i = i - m;
else
i = i - skip;
} else {
/* skip: check if previous character is part of pattern */
if (i > 0 && !STRINGLIB_BLOOM(mask, s[i-1]))
i = i - m;
}
}
}
if (mode != FAST_COUNT)
return -1;
return count;
}
#endif

View File

@ -0,0 +1,175 @@
/* stringlib: find/index implementation */
#ifndef STRINGLIB_FIND_H
#define STRINGLIB_FIND_H
#ifndef STRINGLIB_FASTSEARCH_H
#error must include "stringlib/fastsearch.h" before including this module
#endif
Py_LOCAL_INLINE(Py_ssize_t)
stringlib_find(const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sub, Py_ssize_t sub_len,
Py_ssize_t offset)
{
Py_ssize_t pos;
if (str_len < 0)
return -1;
if (sub_len == 0)
return offset;
pos = fastsearch(str, str_len, sub, sub_len, -1, FAST_SEARCH);
if (pos >= 0)
pos += offset;
return pos;
}
Py_LOCAL_INLINE(Py_ssize_t)
stringlib_rfind(const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sub, Py_ssize_t sub_len,
Py_ssize_t offset)
{
Py_ssize_t pos;
if (str_len < 0)
return -1;
if (sub_len == 0)
return str_len + offset;
pos = fastsearch(str, str_len, sub, sub_len, -1, FAST_RSEARCH);
if (pos >= 0)
pos += offset;
return pos;
}
/* helper macro to fixup start/end slice values */
#define ADJUST_INDICES(start, end, len) \
if (end > len) \
end = len; \
else if (end < 0) { \
end += len; \
if (end < 0) \
end = 0; \
} \
if (start < 0) { \
start += len; \
if (start < 0) \
start = 0; \
}
Py_LOCAL_INLINE(Py_ssize_t)
stringlib_find_slice(const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sub, Py_ssize_t sub_len,
Py_ssize_t start, Py_ssize_t end)
{
ADJUST_INDICES(start, end, str_len);
return stringlib_find(str + start, end - start, sub, sub_len, start);
}
Py_LOCAL_INLINE(Py_ssize_t)
stringlib_rfind_slice(const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sub, Py_ssize_t sub_len,
Py_ssize_t start, Py_ssize_t end)
{
ADJUST_INDICES(start, end, str_len);
return stringlib_rfind(str + start, end - start, sub, sub_len, start);
}
#ifdef STRINGLIB_WANT_CONTAINS_OBJ
Py_LOCAL_INLINE(int)
stringlib_contains_obj(PyObject* str, PyObject* sub)
{
return stringlib_find(
STRINGLIB_STR(str), STRINGLIB_LEN(str),
STRINGLIB_STR(sub), STRINGLIB_LEN(sub), 0
) != -1;
}
#endif /* STRINGLIB_WANT_CONTAINS_OBJ */
/*
This function is a helper for the "find" family (find, rfind, index,
rindex) and for count, startswith and endswith, because they all have
the same behaviour for the arguments.
It does not touch the variables received until it knows everything
is ok.
*/
#define FORMAT_BUFFER_SIZE 50
Py_LOCAL_INLINE(int)
stringlib_parse_args_finds(const char * function_name, PyObject *args,
PyObject **subobj,
Py_ssize_t *start, Py_ssize_t *end)
{
PyObject *tmp_subobj;
Py_ssize_t tmp_start = 0;
Py_ssize_t tmp_end = PY_SSIZE_T_MAX;
PyObject *obj_start=Py_None, *obj_end=Py_None;
char format[FORMAT_BUFFER_SIZE] = "O|OO:";
size_t len = strlen(format);
strncpy(format + len, function_name, FORMAT_BUFFER_SIZE - len - 1);
format[FORMAT_BUFFER_SIZE - 1] = '\0';
if (!PyArg_ParseTuple(args, format, &tmp_subobj, &obj_start, &obj_end))
return 0;
/* To support None in "start" and "end" arguments, meaning
the same as if they were not passed.
*/
if (obj_start != Py_None)
if (!_PyEval_SliceIndex(obj_start, &tmp_start))
return 0;
if (obj_end != Py_None)
if (!_PyEval_SliceIndex(obj_end, &tmp_end))
return 0;
*start = tmp_start;
*end = tmp_end;
*subobj = tmp_subobj;
return 1;
}
#undef FORMAT_BUFFER_SIZE
#if STRINGLIB_IS_UNICODE
/*
Wraps stringlib_parse_args_finds() and additionally ensures that the
first argument is a unicode object.
Note that we receive a pointer to the pointer of the substring object,
so when we create that object in this function we don't DECREF it,
because it continues living in the caller functions (those functions,
after finishing using the substring, must DECREF it).
*/
Py_LOCAL_INLINE(int)
stringlib_parse_args_finds_unicode(const char * function_name, PyObject *args,
PyUnicodeObject **substring,
Py_ssize_t *start, Py_ssize_t *end)
{
PyObject *tmp_substring;
if(stringlib_parse_args_finds(function_name, args, &tmp_substring,
start, end)) {
tmp_substring = PyUnicode_FromObject(tmp_substring);
if (!tmp_substring)
return 0;
*substring = (PyUnicodeObject *)tmp_substring;
return 1;
}
return 0;
}
#endif /* STRINGLIB_IS_UNICODE */
#endif /* STRINGLIB_FIND_H */

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,110 @@
/* stringlib: partition implementation */
#ifndef STRINGLIB_PARTITION_H
#define STRINGLIB_PARTITION_H
#ifndef STRINGLIB_FASTSEARCH_H
#error must include "stringlib/fastsearch.h" before including this module
#endif
Py_LOCAL_INLINE(PyObject*)
stringlib_partition(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
PyObject* sep_obj,
const STRINGLIB_CHAR* sep, Py_ssize_t sep_len)
{
PyObject* out;
Py_ssize_t pos;
if (sep_len == 0) {
PyErr_SetString(PyExc_ValueError, "empty separator");
return NULL;
}
out = PyTuple_New(3);
if (!out)
return NULL;
pos = fastsearch(str, str_len, sep, sep_len, -1, FAST_SEARCH);
if (pos < 0) {
#if STRINGLIB_MUTABLE
PyTuple_SET_ITEM(out, 0, STRINGLIB_NEW(str, str_len));
PyTuple_SET_ITEM(out, 1, STRINGLIB_NEW(NULL, 0));
PyTuple_SET_ITEM(out, 2, STRINGLIB_NEW(NULL, 0));
#else
Py_INCREF(str_obj);
PyTuple_SET_ITEM(out, 0, (PyObject*) str_obj);
Py_INCREF(STRINGLIB_EMPTY);
PyTuple_SET_ITEM(out, 1, (PyObject*) STRINGLIB_EMPTY);
Py_INCREF(STRINGLIB_EMPTY);
PyTuple_SET_ITEM(out, 2, (PyObject*) STRINGLIB_EMPTY);
#endif
return out;
}
PyTuple_SET_ITEM(out, 0, STRINGLIB_NEW(str, pos));
Py_INCREF(sep_obj);
PyTuple_SET_ITEM(out, 1, sep_obj);
pos += sep_len;
PyTuple_SET_ITEM(out, 2, STRINGLIB_NEW(str + pos, str_len - pos));
if (PyErr_Occurred()) {
Py_DECREF(out);
return NULL;
}
return out;
}
Py_LOCAL_INLINE(PyObject*)
stringlib_rpartition(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
PyObject* sep_obj,
const STRINGLIB_CHAR* sep, Py_ssize_t sep_len)
{
PyObject* out;
Py_ssize_t pos;
if (sep_len == 0) {
PyErr_SetString(PyExc_ValueError, "empty separator");
return NULL;
}
out = PyTuple_New(3);
if (!out)
return NULL;
pos = fastsearch(str, str_len, sep, sep_len, -1, FAST_RSEARCH);
if (pos < 0) {
#if STRINGLIB_MUTABLE
PyTuple_SET_ITEM(out, 0, STRINGLIB_NEW(NULL, 0));
PyTuple_SET_ITEM(out, 1, STRINGLIB_NEW(NULL, 0));
PyTuple_SET_ITEM(out, 2, STRINGLIB_NEW(str, str_len));
#else
Py_INCREF(STRINGLIB_EMPTY);
PyTuple_SET_ITEM(out, 0, (PyObject*) STRINGLIB_EMPTY);
Py_INCREF(STRINGLIB_EMPTY);
PyTuple_SET_ITEM(out, 1, (PyObject*) STRINGLIB_EMPTY);
Py_INCREF(str_obj);
PyTuple_SET_ITEM(out, 2, (PyObject*) str_obj);
#endif
return out;
}
PyTuple_SET_ITEM(out, 0, STRINGLIB_NEW(str, pos));
Py_INCREF(sep_obj);
PyTuple_SET_ITEM(out, 1, sep_obj);
pos += sep_len;
PyTuple_SET_ITEM(out, 2, STRINGLIB_NEW(str + pos, str_len - pos));
if (PyErr_Occurred()) {
Py_DECREF(out);
return NULL;
}
return out;
}
#endif

View File

@ -0,0 +1,394 @@
/* stringlib: split implementation */
#ifndef STRINGLIB_SPLIT_H
#define STRINGLIB_SPLIT_H
#ifndef STRINGLIB_FASTSEARCH_H
#error must include "stringlib/fastsearch.h" before including this module
#endif
/* Overallocate the initial list to reduce the number of reallocs for small
split sizes. Eg, "A A A A A A A A A A".split() (10 elements) has three
resizes, to sizes 4, 8, then 16. Most observed string splits are for human
text (roughly 11 words per line) and field delimited data (usually 1-10
fields). For large strings the split algorithms are bandwidth limited
so increasing the preallocation likely will not improve things.*/
#define MAX_PREALLOC 12
/* 5 splits gives 6 elements */
#define PREALLOC_SIZE(maxsplit) \
(maxsplit >= MAX_PREALLOC ? MAX_PREALLOC : maxsplit+1)
#define SPLIT_APPEND(data, left, right) \
sub = STRINGLIB_NEW((data) + (left), \
(right) - (left)); \
if (sub == NULL) \
goto onError; \
if (PyList_Append(list, sub)) { \
Py_DECREF(sub); \
goto onError; \
} \
else \
Py_DECREF(sub);
#define SPLIT_ADD(data, left, right) { \
sub = STRINGLIB_NEW((data) + (left), \
(right) - (left)); \
if (sub == NULL) \
goto onError; \
if (count < MAX_PREALLOC) { \
PyList_SET_ITEM(list, count, sub); \
} else { \
if (PyList_Append(list, sub)) { \
Py_DECREF(sub); \
goto onError; \
} \
else \
Py_DECREF(sub); \
} \
count++; }
/* Always force the list to the expected size. */
#define FIX_PREALLOC_SIZE(list) Py_SIZE(list) = count
Py_LOCAL_INLINE(PyObject *)
stringlib_split_whitespace(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
Py_ssize_t maxcount)
{
Py_ssize_t i, j, count=0;
PyObject *list = PyList_New(PREALLOC_SIZE(maxcount));
PyObject *sub;
if (list == NULL)
return NULL;
i = j = 0;
while (maxcount-- > 0) {
while (i < str_len && STRINGLIB_ISSPACE(str[i]))
i++;
if (i == str_len) break;
j = i; i++;
while (i < str_len && !STRINGLIB_ISSPACE(str[i]))
i++;
#ifndef STRINGLIB_MUTABLE
if (j == 0 && i == str_len && STRINGLIB_CHECK_EXACT(str_obj)) {
/* No whitespace in str_obj, so just use it as list[0] */
Py_INCREF(str_obj);
PyList_SET_ITEM(list, 0, (PyObject *)str_obj);
count++;
break;
}
#endif
SPLIT_ADD(str, j, i);
}
if (i < str_len) {
/* Only occurs when maxcount was reached */
/* Skip any remaining whitespace and copy to end of string */
while (i < str_len && STRINGLIB_ISSPACE(str[i]))
i++;
if (i != str_len)
SPLIT_ADD(str, i, str_len);
}
FIX_PREALLOC_SIZE(list);
return list;
onError:
Py_DECREF(list);
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
stringlib_split_char(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR ch,
Py_ssize_t maxcount)
{
Py_ssize_t i, j, count=0;
PyObject *list = PyList_New(PREALLOC_SIZE(maxcount));
PyObject *sub;
if (list == NULL)
return NULL;
i = j = 0;
while ((j < str_len) && (maxcount-- > 0)) {
for(; j < str_len; j++) {
/* I found that using memchr makes no difference */
if (str[j] == ch) {
SPLIT_ADD(str, i, j);
i = j = j + 1;
break;
}
}
}
#ifndef STRINGLIB_MUTABLE
if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) {
/* ch not in str_obj, so just use str_obj as list[0] */
Py_INCREF(str_obj);
PyList_SET_ITEM(list, 0, (PyObject *)str_obj);
count++;
} else
#endif
if (i <= str_len) {
SPLIT_ADD(str, i, str_len);
}
FIX_PREALLOC_SIZE(list);
return list;
onError:
Py_DECREF(list);
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
stringlib_split(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sep, Py_ssize_t sep_len,
Py_ssize_t maxcount)
{
Py_ssize_t i, j, pos, count=0;
PyObject *list, *sub;
if (sep_len == 0) {
PyErr_SetString(PyExc_ValueError, "empty separator");
return NULL;
}
else if (sep_len == 1)
return stringlib_split_char(str_obj, str, str_len, sep[0], maxcount);
list = PyList_New(PREALLOC_SIZE(maxcount));
if (list == NULL)
return NULL;
i = j = 0;
while (maxcount-- > 0) {
pos = fastsearch(str+i, str_len-i, sep, sep_len, -1, FAST_SEARCH);
if (pos < 0)
break;
j = i + pos;
SPLIT_ADD(str, i, j);
i = j + sep_len;
}
#ifndef STRINGLIB_MUTABLE
if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) {
/* No match in str_obj, so just use it as list[0] */
Py_INCREF(str_obj);
PyList_SET_ITEM(list, 0, (PyObject *)str_obj);
count++;
} else
#endif
{
SPLIT_ADD(str, i, str_len);
}
FIX_PREALLOC_SIZE(list);
return list;
onError:
Py_DECREF(list);
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
stringlib_rsplit_whitespace(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
Py_ssize_t maxcount)
{
Py_ssize_t i, j, count=0;
PyObject *list = PyList_New(PREALLOC_SIZE(maxcount));
PyObject *sub;
if (list == NULL)
return NULL;
i = j = str_len - 1;
while (maxcount-- > 0) {
while (i >= 0 && STRINGLIB_ISSPACE(str[i]))
i--;
if (i < 0) break;
j = i; i--;
while (i >= 0 && !STRINGLIB_ISSPACE(str[i]))
i--;
#ifndef STRINGLIB_MUTABLE
if (j == str_len - 1 && i < 0 && STRINGLIB_CHECK_EXACT(str_obj)) {
/* No whitespace in str_obj, so just use it as list[0] */
Py_INCREF(str_obj);
PyList_SET_ITEM(list, 0, (PyObject *)str_obj);
count++;
break;
}
#endif
SPLIT_ADD(str, i + 1, j + 1);
}
if (i >= 0) {
/* Only occurs when maxcount was reached */
/* Skip any remaining whitespace and copy to beginning of string */
while (i >= 0 && STRINGLIB_ISSPACE(str[i]))
i--;
if (i >= 0)
SPLIT_ADD(str, 0, i + 1);
}
FIX_PREALLOC_SIZE(list);
if (PyList_Reverse(list) < 0)
goto onError;
return list;
onError:
Py_DECREF(list);
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
stringlib_rsplit_char(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR ch,
Py_ssize_t maxcount)
{
Py_ssize_t i, j, count=0;
PyObject *list = PyList_New(PREALLOC_SIZE(maxcount));
PyObject *sub;
if (list == NULL)
return NULL;
i = j = str_len - 1;
while ((i >= 0) && (maxcount-- > 0)) {
for(; i >= 0; i--) {
if (str[i] == ch) {
SPLIT_ADD(str, i + 1, j + 1);
j = i = i - 1;
break;
}
}
}
#ifndef STRINGLIB_MUTABLE
if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) {
/* ch not in str_obj, so just use str_obj as list[0] */
Py_INCREF(str_obj);
PyList_SET_ITEM(list, 0, (PyObject *)str_obj);
count++;
} else
#endif
if (j >= -1) {
SPLIT_ADD(str, 0, j + 1);
}
FIX_PREALLOC_SIZE(list);
if (PyList_Reverse(list) < 0)
goto onError;
return list;
onError:
Py_DECREF(list);
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
stringlib_rsplit(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
const STRINGLIB_CHAR* sep, Py_ssize_t sep_len,
Py_ssize_t maxcount)
{
Py_ssize_t j, pos, count=0;
PyObject *list, *sub;
if (sep_len == 0) {
PyErr_SetString(PyExc_ValueError, "empty separator");
return NULL;
}
else if (sep_len == 1)
return stringlib_rsplit_char(str_obj, str, str_len, sep[0], maxcount);
list = PyList_New(PREALLOC_SIZE(maxcount));
if (list == NULL)
return NULL;
j = str_len;
while (maxcount-- > 0) {
pos = fastsearch(str, j, sep, sep_len, -1, FAST_RSEARCH);
if (pos < 0)
break;
SPLIT_ADD(str, pos + sep_len, j);
j = pos;
}
#ifndef STRINGLIB_MUTABLE
if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) {
/* No match in str_obj, so just use it as list[0] */
Py_INCREF(str_obj);
PyList_SET_ITEM(list, 0, (PyObject *)str_obj);
count++;
} else
#endif
{
SPLIT_ADD(str, 0, j);
}
FIX_PREALLOC_SIZE(list);
if (PyList_Reverse(list) < 0)
goto onError;
return list;
onError:
Py_DECREF(list);
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
stringlib_splitlines(PyObject* str_obj,
const STRINGLIB_CHAR* str, Py_ssize_t str_len,
int keepends)
{
/* This does not use the preallocated list because splitlines is
usually run with hundreds of newlines. The overhead of
switching between PyList_SET_ITEM and append causes about a
2-3% slowdown for that common case. A smarter implementation
could move the if check out, so the SET_ITEMs are done first
and the appends only done when the prealloc buffer is full.
That's too much work for little gain.*/
register Py_ssize_t i;
register Py_ssize_t j;
PyObject *list = PyList_New(0);
PyObject *sub;
if (list == NULL)
return NULL;
for (i = j = 0; i < str_len; ) {
Py_ssize_t eol;
/* Find a line and append it */
while (i < str_len && !STRINGLIB_ISLINEBREAK(str[i]))
i++;
/* Skip the line break reading CRLF as one line break */
eol = i;
if (i < str_len) {
if (str[i] == '\r' && i + 1 < str_len && str[i+1] == '\n')
i += 2;
else
i++;
if (keepends)
eol = i;
}
#ifndef STRINGLIB_MUTABLE
if (j == 0 && eol == str_len && STRINGLIB_CHECK_EXACT(str_obj)) {
/* No linebreak in str_obj, so just use it as list[0] */
if (PyList_Append(list, str_obj))
goto onError;
break;
}
#endif
SPLIT_APPEND(str, j, eol);
j = i;
}
return list;
onError:
Py_DECREF(list);
return NULL;
}
#endif

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,33 @@
#ifndef STRINGLIB_STRINGDEFS_H
#define STRINGLIB_STRINGDEFS_H
/* this is sort of a hack. there's at least one place (formatting
floats) where some stringlib code takes a different path if it's
compiled as unicode. */
#define STRINGLIB_IS_UNICODE 0
#define STRINGLIB_OBJECT PyStringObject
#define STRINGLIB_CHAR char
#define STRINGLIB_TYPE_NAME "string"
#define STRINGLIB_PARSE_CODE "S"
#define STRINGLIB_EMPTY nullstring
#define STRINGLIB_ISSPACE Py_ISSPACE
#define STRINGLIB_ISLINEBREAK(x) ((x == '\n') || (x == '\r'))
#define STRINGLIB_ISDECIMAL(x) ((x >= '0') && (x <= '9'))
#define STRINGLIB_TODECIMAL(x) (STRINGLIB_ISDECIMAL(x) ? (x - '0') : -1)
#define STRINGLIB_TOUPPER Py_TOUPPER
#define STRINGLIB_TOLOWER Py_TOLOWER
#define STRINGLIB_FILL memset
#define STRINGLIB_STR PyString_AS_STRING
#define STRINGLIB_LEN PyString_GET_SIZE
#define STRINGLIB_NEW PyString_FromStringAndSize
#define STRINGLIB_RESIZE _PyString_Resize
#define STRINGLIB_CHECK PyString_Check
#define STRINGLIB_CHECK_EXACT PyString_CheckExact
#define STRINGLIB_TOSTR PyObject_Str
#define STRINGLIB_GROUPING _PyString_InsertThousandsGrouping
#define STRINGLIB_GROUPING_LOCALE _PyString_InsertThousandsGroupingLocale
#define STRINGLIB_WANT_CONTAINS_OBJ 1
#endif /* !STRINGLIB_STRINGDEFS_H */

View File

@ -0,0 +1,264 @@
/* NOTE: this API is -ONLY- for use with single byte character strings. */
/* Do not use it with Unicode. */
/* the more complicated methods. parts of these should be pulled out into the
shared code in bytes_methods.c to cut down on duplicate code bloat. */
PyDoc_STRVAR(expandtabs__doc__,
"B.expandtabs([tabsize]) -> copy of B\n\
\n\
Return a copy of B where all tab characters are expanded using spaces.\n\
If tabsize is not given, a tab size of 8 characters is assumed.");
static PyObject*
stringlib_expandtabs(PyObject *self, PyObject *args)
{
const char *e, *p;
char *q;
Py_ssize_t i, j;
PyObject *u;
int tabsize = 8;
if (!PyArg_ParseTuple(args, "|i:expandtabs", &tabsize))
return NULL;
/* First pass: determine size of output string */
i = j = 0;
e = STRINGLIB_STR(self) + STRINGLIB_LEN(self);
for (p = STRINGLIB_STR(self); p < e; p++) {
if (*p == '\t') {
if (tabsize > 0) {
Py_ssize_t incr = tabsize - (j % tabsize);
if (j > PY_SSIZE_T_MAX - incr)
goto overflow;
j += incr;
}
}
else {
if (j > PY_SSIZE_T_MAX - 1)
goto overflow;
j++;
if (*p == '\n' || *p == '\r') {
if (i > PY_SSIZE_T_MAX - j)
goto overflow;
i += j;
j = 0;
}
}
}
if (i > PY_SSIZE_T_MAX - j)
goto overflow;
/* Second pass: create output string and fill it */
u = STRINGLIB_NEW(NULL, i + j);
if (!u)
return NULL;
j = 0;
q = STRINGLIB_STR(u);
for (p = STRINGLIB_STR(self); p < e; p++) {
if (*p == '\t') {
if (tabsize > 0) {
i = tabsize - (j % tabsize);
j += i;
while (i--)
*q++ = ' ';
}
}
else {
j++;
*q++ = *p;
if (*p == '\n' || *p == '\r')
j = 0;
}
}
return u;
overflow:
PyErr_SetString(PyExc_OverflowError, "result too long");
return NULL;
}
Py_LOCAL_INLINE(PyObject *)
pad(PyObject *self, Py_ssize_t left, Py_ssize_t right, char fill)
{
PyObject *u;
if (left < 0)
left = 0;
if (right < 0)
right = 0;
if (left == 0 && right == 0 && STRINGLIB_CHECK_EXACT(self)) {
#if STRINGLIB_MUTABLE
/* We're defined as returning a copy; If the object is mutable
* that means we must make an identical copy. */
return STRINGLIB_NEW(STRINGLIB_STR(self), STRINGLIB_LEN(self));
#else
Py_INCREF(self);
return (PyObject *)self;
#endif /* STRINGLIB_MUTABLE */
}
u = STRINGLIB_NEW(NULL,
left + STRINGLIB_LEN(self) + right);
if (u) {
if (left)
memset(STRINGLIB_STR(u), fill, left);
Py_MEMCPY(STRINGLIB_STR(u) + left,
STRINGLIB_STR(self),
STRINGLIB_LEN(self));
if (right)
memset(STRINGLIB_STR(u) + left + STRINGLIB_LEN(self),
fill, right);
}
return u;
}
PyDoc_STRVAR(ljust__doc__,
"B.ljust(width[, fillchar]) -> copy of B\n"
"\n"
"Return B left justified in a string of length width. Padding is\n"
"done using the specified fill character (default is a space).");
static PyObject *
stringlib_ljust(PyObject *self, PyObject *args)
{
Py_ssize_t width;
char fillchar = ' ';
if (!PyArg_ParseTuple(args, "n|c:ljust", &width, &fillchar))
return NULL;
if (STRINGLIB_LEN(self) >= width && STRINGLIB_CHECK_EXACT(self)) {
#if STRINGLIB_MUTABLE
/* We're defined as returning a copy; If the object is mutable
* that means we must make an identical copy. */
return STRINGLIB_NEW(STRINGLIB_STR(self), STRINGLIB_LEN(self));
#else
Py_INCREF(self);
return (PyObject*) self;
#endif
}
return pad(self, 0, width - STRINGLIB_LEN(self), fillchar);
}
PyDoc_STRVAR(rjust__doc__,
"B.rjust(width[, fillchar]) -> copy of B\n"
"\n"
"Return B right justified in a string of length width. Padding is\n"
"done using the specified fill character (default is a space)");
static PyObject *
stringlib_rjust(PyObject *self, PyObject *args)
{
Py_ssize_t width;
char fillchar = ' ';
if (!PyArg_ParseTuple(args, "n|c:rjust", &width, &fillchar))
return NULL;
if (STRINGLIB_LEN(self) >= width && STRINGLIB_CHECK_EXACT(self)) {
#if STRINGLIB_MUTABLE
/* We're defined as returning a copy; If the object is mutable
* that means we must make an identical copy. */
return STRINGLIB_NEW(STRINGLIB_STR(self), STRINGLIB_LEN(self));
#else
Py_INCREF(self);
return (PyObject*) self;
#endif
}
return pad(self, width - STRINGLIB_LEN(self), 0, fillchar);
}
PyDoc_STRVAR(center__doc__,
"B.center(width[, fillchar]) -> copy of B\n"
"\n"
"Return B centered in a string of length width. Padding is\n"
"done using the specified fill character (default is a space).");
static PyObject *
stringlib_center(PyObject *self, PyObject *args)
{
Py_ssize_t marg, left;
Py_ssize_t width;
char fillchar = ' ';
if (!PyArg_ParseTuple(args, "n|c:center", &width, &fillchar))
return NULL;
if (STRINGLIB_LEN(self) >= width && STRINGLIB_CHECK_EXACT(self)) {
#if STRINGLIB_MUTABLE
/* We're defined as returning a copy; If the object is mutable
* that means we must make an identical copy. */
return STRINGLIB_NEW(STRINGLIB_STR(self), STRINGLIB_LEN(self));
#else
Py_INCREF(self);
return (PyObject*) self;
#endif
}
marg = width - STRINGLIB_LEN(self);
left = marg / 2 + (marg & width & 1);
return pad(self, left, marg - left, fillchar);
}
PyDoc_STRVAR(zfill__doc__,
"B.zfill(width) -> copy of B\n"
"\n"
"Pad a numeric string B with zeros on the left, to fill a field\n"
"of the specified width. B is never truncated.");
static PyObject *
stringlib_zfill(PyObject *self, PyObject *args)
{
Py_ssize_t fill;
PyObject *s;
char *p;
Py_ssize_t width;
if (!PyArg_ParseTuple(args, "n:zfill", &width))
return NULL;
if (STRINGLIB_LEN(self) >= width) {
if (STRINGLIB_CHECK_EXACT(self)) {
#if STRINGLIB_MUTABLE
/* We're defined as returning a copy; If the object is mutable
* that means we must make an identical copy. */
return STRINGLIB_NEW(STRINGLIB_STR(self), STRINGLIB_LEN(self));
#else
Py_INCREF(self);
return (PyObject*) self;
#endif
}
else
return STRINGLIB_NEW(
STRINGLIB_STR(self),
STRINGLIB_LEN(self)
);
}
fill = width - STRINGLIB_LEN(self);
s = pad(self, fill, 0, '0');
if (s == NULL)
return NULL;
p = STRINGLIB_STR(s);
if (p[fill] == '+' || p[fill] == '-') {
/* move sign to beginning of string */
p[0] = p[fill];
p[fill] = '0';
}
return (PyObject*) s;
}

View File

@ -0,0 +1,37 @@
#ifndef STRINGLIB_UNICODEDEFS_H
#define STRINGLIB_UNICODEDEFS_H
/* this is sort of a hack. there's at least one place (formatting
floats) where some stringlib code takes a different path if it's
compiled as unicode. */
#define STRINGLIB_IS_UNICODE 1
#define STRINGLIB_OBJECT PyUnicodeObject
#define STRINGLIB_CHAR Py_UNICODE
#define STRINGLIB_TYPE_NAME "unicode"
#define STRINGLIB_PARSE_CODE "U"
#define STRINGLIB_EMPTY unicode_empty
#define STRINGLIB_ISSPACE Py_UNICODE_ISSPACE
#define STRINGLIB_ISLINEBREAK BLOOM_LINEBREAK
#define STRINGLIB_ISDECIMAL Py_UNICODE_ISDECIMAL
#define STRINGLIB_TODECIMAL Py_UNICODE_TODECIMAL
#define STRINGLIB_TOUPPER Py_UNICODE_TOUPPER
#define STRINGLIB_TOLOWER Py_UNICODE_TOLOWER
#define STRINGLIB_FILL Py_UNICODE_FILL
#define STRINGLIB_STR PyUnicode_AS_UNICODE
#define STRINGLIB_LEN PyUnicode_GET_SIZE
#define STRINGLIB_NEW PyUnicode_FromUnicode
#define STRINGLIB_RESIZE PyUnicode_Resize
#define STRINGLIB_CHECK PyUnicode_Check
#define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact
#define STRINGLIB_GROUPING _PyUnicode_InsertThousandsGrouping
#if PY_VERSION_HEX < 0x03000000
#define STRINGLIB_TOSTR PyObject_Unicode
#else
#define STRINGLIB_TOSTR PyObject_Str
#endif
#define STRINGLIB_WANT_CONTAINS_OBJ 1
#endif /* !STRINGLIB_UNICODEDEFS_H */

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,540 @@
/* Implementation helper: a struct that looks like a tuple. See timemodule
and posixmodule for example uses. */
#include "Python.h"
#include "structmember.h"
#include "structseq.h"
static char visible_length_key[] = "n_sequence_fields";
static char real_length_key[] = "n_fields";
static char unnamed_fields_key[] = "n_unnamed_fields";
/* Fields with this name have only a field index, not a field name.
They are only allowed for indices < n_visible_fields. */
char *PyStructSequence_UnnamedField = "unnamed field";
#define VISIBLE_SIZE(op) Py_SIZE(op)
#define VISIBLE_SIZE_TP(tp) PyInt_AsLong( \
PyDict_GetItemString((tp)->tp_dict, visible_length_key))
#define REAL_SIZE_TP(tp) PyInt_AsLong( \
PyDict_GetItemString((tp)->tp_dict, real_length_key))
#define REAL_SIZE(op) REAL_SIZE_TP(Py_TYPE(op))
#define UNNAMED_FIELDS_TP(tp) PyInt_AsLong( \
PyDict_GetItemString((tp)->tp_dict, unnamed_fields_key))
#define UNNAMED_FIELDS(op) UNNAMED_FIELDS_TP(Py_TYPE(op))
PyObject *
PyStructSequence_New(PyTypeObject *type)
{
PyStructSequence *obj;
obj = PyObject_New(PyStructSequence, type);
if (obj == NULL)
return NULL;
Py_SIZE(obj) = VISIBLE_SIZE_TP(type);
return (PyObject*) obj;
}
static void
structseq_dealloc(PyStructSequence *obj)
{
Py_ssize_t i, size;
size = REAL_SIZE(obj);
for (i = 0; i < size; ++i) {
Py_XDECREF(obj->ob_item[i]);
}
PyObject_Del(obj);
}
static Py_ssize_t
structseq_length(PyStructSequence *obj)
{
return VISIBLE_SIZE(obj);
}
static PyObject*
structseq_item(PyStructSequence *obj, Py_ssize_t i)
{
if (i < 0 || i >= VISIBLE_SIZE(obj)) {
PyErr_SetString(PyExc_IndexError, "tuple index out of range");
return NULL;
}
Py_INCREF(obj->ob_item[i]);
return obj->ob_item[i];
}
static PyObject*
structseq_slice(PyStructSequence *obj, Py_ssize_t low, Py_ssize_t high)
{
PyTupleObject *np;
Py_ssize_t i;
if (low < 0)
low = 0;
if (high > VISIBLE_SIZE(obj))
high = VISIBLE_SIZE(obj);
if (high < low)
high = low;
np = (PyTupleObject *)PyTuple_New(high-low);
if (np == NULL)
return NULL;
for(i = low; i < high; ++i) {
PyObject *v = obj->ob_item[i];
Py_INCREF(v);
PyTuple_SET_ITEM(np, i-low, v);
}
return (PyObject *) np;
}
static PyObject *
structseq_subscript(PyStructSequence *self, PyObject *item)
{
if (PyIndex_Check(item)) {
Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
if (i == -1 && PyErr_Occurred())
return NULL;
if (i < 0)
i += VISIBLE_SIZE(self);
if (i < 0 || i >= VISIBLE_SIZE(self)) {
PyErr_SetString(PyExc_IndexError,
"tuple index out of range");
return NULL;
}
Py_INCREF(self->ob_item[i]);
return self->ob_item[i];
}
else if (PySlice_Check(item)) {
Py_ssize_t start, stop, step, slicelen, cur, i;
PyObject *result;
if (PySlice_GetIndicesEx((PySliceObject *)item,
VISIBLE_SIZE(self), &start, &stop,
&step, &slicelen) < 0) {
return NULL;
}
if (slicelen <= 0)
return PyTuple_New(0);
result = PyTuple_New(slicelen);
if (result == NULL)
return NULL;
for (cur = start, i = 0; i < slicelen;
cur += step, i++) {
PyObject *v = self->ob_item[cur];
Py_INCREF(v);
PyTuple_SET_ITEM(result, i, v);
}
return result;
}
else {
PyErr_SetString(PyExc_TypeError,
"structseq index must be integer");
return NULL;
}
}
static PyObject *
structseq_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
PyObject *arg = NULL;
PyObject *dict = NULL;
PyObject *ob;
PyStructSequence *res = NULL;
Py_ssize_t len, min_len, max_len, i, n_unnamed_fields;
static char *kwlist[] = {"sequence", "dict", 0};
if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:structseq",
kwlist, &arg, &dict))
return NULL;
arg = PySequence_Fast(arg, "constructor requires a sequence");
if (!arg) {
return NULL;
}
if (dict && !PyDict_Check(dict)) {
PyErr_Format(PyExc_TypeError,
"%.500s() takes a dict as second arg, if any",
type->tp_name);
Py_DECREF(arg);
return NULL;
}
len = PySequence_Fast_GET_SIZE(arg);
min_len = VISIBLE_SIZE_TP(type);
max_len = REAL_SIZE_TP(type);
n_unnamed_fields = UNNAMED_FIELDS_TP(type);
if (min_len != max_len) {
if (len < min_len) {
PyErr_Format(PyExc_TypeError,
"%.500s() takes an at least %zd-sequence (%zd-sequence given)",
type->tp_name, min_len, len);
Py_DECREF(arg);
return NULL;
}
if (len > max_len) {
PyErr_Format(PyExc_TypeError,
"%.500s() takes an at most %zd-sequence (%zd-sequence given)",
type->tp_name, max_len, len);
Py_DECREF(arg);
return NULL;
}
}
else {
if (len != min_len) {
PyErr_Format(PyExc_TypeError,
"%.500s() takes a %zd-sequence (%zd-sequence given)",
type->tp_name, min_len, len);
Py_DECREF(arg);
return NULL;
}
}
res = (PyStructSequence*) PyStructSequence_New(type);
if (res == NULL) {
Py_DECREF(arg);
return NULL;
}
for (i = 0; i < len; ++i) {
PyObject *v = PySequence_Fast_GET_ITEM(arg, i);
Py_INCREF(v);
res->ob_item[i] = v;
}
for (; i < max_len; ++i) {
if (dict && (ob = PyDict_GetItemString(
dict, type->tp_members[i-n_unnamed_fields].name))) {
}
else {
ob = Py_None;
}
Py_INCREF(ob);
res->ob_item[i] = ob;
}
Py_DECREF(arg);
return (PyObject*) res;
}
static PyObject *
make_tuple(PyStructSequence *obj)
{
return structseq_slice(obj, 0, VISIBLE_SIZE(obj));
}
static PyObject *
structseq_repr(PyStructSequence *obj)
{
/* buffer and type size were chosen well considered. */
#define REPR_BUFFER_SIZE 512
#define TYPE_MAXSIZE 100
PyObject *tup;
PyTypeObject *typ = Py_TYPE(obj);
int i, removelast = 0;
Py_ssize_t len;
char buf[REPR_BUFFER_SIZE];
char *endofbuf, *pbuf = buf;
/* pointer to end of writeable buffer; safes space for "...)\0" */
endofbuf= &buf[REPR_BUFFER_SIZE-5];
if ((tup = make_tuple(obj)) == NULL) {
return NULL;
}
/* "typename(", limited to TYPE_MAXSIZE */
len = strlen(typ->tp_name) > TYPE_MAXSIZE ? TYPE_MAXSIZE :
strlen(typ->tp_name);
strncpy(pbuf, typ->tp_name, len);
pbuf += len;
*pbuf++ = '(';
for (i=0; i < VISIBLE_SIZE(obj); i++) {
PyObject *val, *repr;
char *cname, *crepr;
cname = typ->tp_members[i].name;
val = PyTuple_GetItem(tup, i);
if (cname == NULL || val == NULL) {
return NULL;
}
repr = PyObject_Repr(val);
if (repr == NULL) {
Py_DECREF(tup);
return NULL;
}
crepr = PyString_AsString(repr);
if (crepr == NULL) {
Py_DECREF(tup);
Py_DECREF(repr);
return NULL;
}
/* + 3: keep space for "=" and ", " */
len = strlen(cname) + strlen(crepr) + 3;
if ((pbuf+len) <= endofbuf) {
strcpy(pbuf, cname);
pbuf += strlen(cname);
*pbuf++ = '=';
strcpy(pbuf, crepr);
pbuf += strlen(crepr);
*pbuf++ = ',';
*pbuf++ = ' ';
removelast = 1;
Py_DECREF(repr);
}
else {
strcpy(pbuf, "...");
pbuf += 3;
removelast = 0;
Py_DECREF(repr);
break;
}
}
Py_DECREF(tup);
if (removelast) {
/* overwrite last ", " */
pbuf-=2;
}
*pbuf++ = ')';
*pbuf = '\0';
return PyString_FromString(buf);
}
static PyObject *
structseq_concat(PyStructSequence *obj, PyObject *b)
{
PyObject *tup, *result;
tup = make_tuple(obj);
result = PySequence_Concat(tup, b);
Py_DECREF(tup);
return result;
}
static PyObject *
structseq_repeat(PyStructSequence *obj, Py_ssize_t n)
{
PyObject *tup, *result;
tup = make_tuple(obj);
result = PySequence_Repeat(tup, n);
Py_DECREF(tup);
return result;
}
static int
structseq_contains(PyStructSequence *obj, PyObject *o)
{
PyObject *tup;
int result;
tup = make_tuple(obj);
if (!tup)
return -1;
result = PySequence_Contains(tup, o);
Py_DECREF(tup);
return result;
}
static long
structseq_hash(PyObject *obj)
{
PyObject *tup;
long result;
tup = make_tuple((PyStructSequence*) obj);
if (!tup)
return -1;
result = PyObject_Hash(tup);
Py_DECREF(tup);
return result;
}
static PyObject *
structseq_richcompare(PyObject *obj, PyObject *o2, int op)
{
PyObject *tup, *result;
tup = make_tuple((PyStructSequence*) obj);
result = PyObject_RichCompare(tup, o2, op);
Py_DECREF(tup);
return result;
}
static PyObject *
structseq_reduce(PyStructSequence* self)
{
PyObject* tup;
PyObject* dict;
PyObject* result;
Py_ssize_t n_fields, n_visible_fields, n_unnamed_fields;
int i;
n_fields = REAL_SIZE(self);
n_visible_fields = VISIBLE_SIZE(self);
n_unnamed_fields = UNNAMED_FIELDS(self);
tup = PyTuple_New(n_visible_fields);
if (!tup) {
return NULL;
}
dict = PyDict_New();
if (!dict) {
Py_DECREF(tup);
return NULL;
}
for (i = 0; i < n_visible_fields; i++) {
Py_INCREF(self->ob_item[i]);
PyTuple_SET_ITEM(tup, i, self->ob_item[i]);
}
for (; i < n_fields; i++) {
char *n = Py_TYPE(self)->tp_members[i-n_unnamed_fields].name;
PyDict_SetItemString(dict, n,
self->ob_item[i]);
}
result = Py_BuildValue("(O(OO))", Py_TYPE(self), tup, dict);
Py_DECREF(tup);
Py_DECREF(dict);
return result;
}
static PySequenceMethods structseq_as_sequence = {
(lenfunc)structseq_length,
(binaryfunc)structseq_concat, /* sq_concat */
(ssizeargfunc)structseq_repeat, /* sq_repeat */
(ssizeargfunc)structseq_item, /* sq_item */
(ssizessizeargfunc)structseq_slice, /* sq_slice */
0, /* sq_ass_item */
0, /* sq_ass_slice */
(objobjproc)structseq_contains, /* sq_contains */
};
static PyMappingMethods structseq_as_mapping = {
(lenfunc)structseq_length,
(binaryfunc)structseq_subscript,
};
static PyMethodDef structseq_methods[] = {
{"__reduce__", (PyCFunction)structseq_reduce,
METH_NOARGS, NULL},
{NULL, NULL}
};
static PyTypeObject _struct_sequence_template = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
NULL, /* tp_name */
0, /* tp_basicsize */
0, /* tp_itemsize */
(destructor)structseq_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
(reprfunc)structseq_repr, /* tp_repr */
0, /* tp_as_number */
&structseq_as_sequence, /* tp_as_sequence */
&structseq_as_mapping, /* tp_as_mapping */
structseq_hash, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
NULL, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
structseq_richcompare, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
structseq_methods, /* tp_methods */
NULL, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
structseq_new, /* tp_new */
};
void
PyStructSequence_InitType(PyTypeObject *type, PyStructSequence_Desc *desc)
{
PyObject *dict;
PyMemberDef* members;
int n_members, n_unnamed_members, i, k;
#ifdef Py_TRACE_REFS
/* if the type object was chained, unchain it first
before overwriting its storage */
if (type->_ob_next) {
_Py_ForgetReference((PyObject*)type);
}
#endif
n_unnamed_members = 0;
for (i = 0; desc->fields[i].name != NULL; ++i)
if (desc->fields[i].name == PyStructSequence_UnnamedField)
n_unnamed_members++;
n_members = i;
memcpy(type, &_struct_sequence_template, sizeof(PyTypeObject));
type->tp_name = desc->name;
type->tp_doc = desc->doc;
type->tp_basicsize = sizeof(PyStructSequence)+
sizeof(PyObject*)*(n_members-1);
type->tp_itemsize = 0;
members = PyMem_NEW(PyMemberDef, n_members-n_unnamed_members+1);
if (members == NULL)
return;
for (i = k = 0; i < n_members; ++i) {
if (desc->fields[i].name == PyStructSequence_UnnamedField)
continue;
members[k].name = desc->fields[i].name;
members[k].type = T_OBJECT;
members[k].offset = offsetof(PyStructSequence, ob_item)
+ i * sizeof(PyObject*);
members[k].flags = READONLY;
members[k].doc = desc->fields[i].doc;
k++;
}
members[k].name = NULL;
type->tp_members = members;
if (PyType_Ready(type) < 0)
return;
Py_INCREF(type);
dict = type->tp_dict;
#define SET_DICT_FROM_INT(key, value) \
do { \
PyObject *v = PyInt_FromLong((long) value); \
if (v != NULL) { \
PyDict_SetItemString(dict, key, v); \
Py_DECREF(v); \
} \
} while (0)
SET_DICT_FROM_INT(visible_length_key, desc->n_in_sequence);
SET_DICT_FROM_INT(real_length_key, n_members);
SET_DICT_FROM_INT(unnamed_fields_key, n_unnamed_members);
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,215 @@
/*
Unicode character type helpers.
Written by Marc-Andre Lemburg (mal@lemburg.com).
Modified for Python 2.0 by Fredrik Lundh (fredrik@pythonware.com)
Copyright (c) Corporation for National Research Initiatives.
*/
#include "Python.h"
#include "unicodeobject.h"
#define ALPHA_MASK 0x01
#define DECIMAL_MASK 0x02
#define DIGIT_MASK 0x04
#define LOWER_MASK 0x08
#define LINEBREAK_MASK 0x10
#define SPACE_MASK 0x20
#define TITLE_MASK 0x40
#define UPPER_MASK 0x80
#define NODELTA_MASK 0x100
#define NUMERIC_MASK 0x200
typedef struct {
const Py_UNICODE upper;
const Py_UNICODE lower;
const Py_UNICODE title;
const unsigned char decimal;
const unsigned char digit;
const unsigned short flags;
} _PyUnicode_TypeRecord;
#include "unicodetype_db.h"
static const _PyUnicode_TypeRecord *
gettyperecord(Py_UNICODE code)
{
int index;
#ifdef Py_UNICODE_WIDE
if (code >= 0x110000)
index = 0;
else
#endif
{
index = index1[(code>>SHIFT)];
index = index2[(index<<SHIFT)+(code&((1<<SHIFT)-1))];
}
return &_PyUnicode_TypeRecords[index];
}
/* Returns the titlecase Unicode characters corresponding to ch or just
ch if no titlecase mapping is known. */
Py_UNICODE _PyUnicode_ToTitlecase(register Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
int delta = ctype->title;
if (ctype->flags & NODELTA_MASK)
return delta;
if (delta >= 32768)
delta -= 65536;
return ch + delta;
}
/* Returns 1 for Unicode characters having the category 'Lt', 0
otherwise. */
int _PyUnicode_IsTitlecase(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & TITLE_MASK) != 0;
}
/* Returns the integer decimal (0-9) for Unicode characters having
this property, -1 otherwise. */
int _PyUnicode_ToDecimalDigit(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & DECIMAL_MASK) ? ctype->decimal : -1;
}
int _PyUnicode_IsDecimalDigit(Py_UNICODE ch)
{
if (_PyUnicode_ToDecimalDigit(ch) < 0)
return 0;
return 1;
}
/* Returns the integer digit (0-9) for Unicode characters having
this property, -1 otherwise. */
int _PyUnicode_ToDigit(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & DIGIT_MASK) ? ctype->digit : -1;
}
int _PyUnicode_IsDigit(Py_UNICODE ch)
{
if (_PyUnicode_ToDigit(ch) < 0)
return 0;
return 1;
}
/* Returns the numeric value as double for Unicode characters having
this property, -1.0 otherwise. */
int _PyUnicode_IsNumeric(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & NUMERIC_MASK) != 0;
}
#ifndef WANT_WCTYPE_FUNCTIONS
/* Returns 1 for Unicode characters having the category 'Ll', 0
otherwise. */
int _PyUnicode_IsLowercase(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & LOWER_MASK) != 0;
}
/* Returns 1 for Unicode characters having the category 'Lu', 0
otherwise. */
int _PyUnicode_IsUppercase(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & UPPER_MASK) != 0;
}
/* Returns the uppercase Unicode characters corresponding to ch or just
ch if no uppercase mapping is known. */
Py_UNICODE _PyUnicode_ToUppercase(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
int delta = ctype->upper;
if (ctype->flags & NODELTA_MASK)
return delta;
if (delta >= 32768)
delta -= 65536;
return ch + delta;
}
/* Returns the lowercase Unicode characters corresponding to ch or just
ch if no lowercase mapping is known. */
Py_UNICODE _PyUnicode_ToLowercase(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
int delta = ctype->lower;
if (ctype->flags & NODELTA_MASK)
return delta;
if (delta >= 32768)
delta -= 65536;
return ch + delta;
}
/* Returns 1 for Unicode characters having the category 'Ll', 'Lu', 'Lt',
'Lo' or 'Lm', 0 otherwise. */
int _PyUnicode_IsAlpha(Py_UNICODE ch)
{
const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
return (ctype->flags & ALPHA_MASK) != 0;
}
#else
/* Export the interfaces using the wchar_t type for portability
reasons: */
int _PyUnicode_IsLowercase(Py_UNICODE ch)
{
return iswlower(ch);
}
int _PyUnicode_IsUppercase(Py_UNICODE ch)
{
return iswupper(ch);
}
Py_UNICODE _PyUnicode_ToLowercase(Py_UNICODE ch)
{
return towlower(ch);
}
Py_UNICODE _PyUnicode_ToUppercase(Py_UNICODE ch)
{
return towupper(ch);
}
int _PyUnicode_IsAlpha(Py_UNICODE ch)
{
return iswalpha(ch);
}
#endif

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,981 @@
#include "Python.h"
#include "structmember.h"
#define GET_WEAKREFS_LISTPTR(o) \
((PyWeakReference **) PyObject_GET_WEAKREFS_LISTPTR(o))
Py_ssize_t
_PyWeakref_GetWeakrefCount(PyWeakReference *head)
{
Py_ssize_t count = 0;
while (head != NULL) {
++count;
head = head->wr_next;
}
return count;
}
static void
init_weakref(PyWeakReference *self, PyObject *ob, PyObject *callback)
{
self->hash = -1;
self->wr_object = ob;
Py_XINCREF(callback);
self->wr_callback = callback;
}
static PyWeakReference *
new_weakref(PyObject *ob, PyObject *callback)
{
PyWeakReference *result;
result = PyObject_GC_New(PyWeakReference, &_PyWeakref_RefType);
if (result) {
init_weakref(result, ob, callback);
PyObject_GC_Track(result);
}
return result;
}
/* This function clears the passed-in reference and removes it from the
* list of weak references for the referent. This is the only code that
* removes an item from the doubly-linked list of weak references for an
* object; it is also responsible for clearing the callback slot.
*/
static void
clear_weakref(PyWeakReference *self)
{
PyObject *callback = self->wr_callback;
if (self->wr_object != Py_None) {
PyWeakReference **list = GET_WEAKREFS_LISTPTR(self->wr_object);
if (*list == self)
/* If 'self' is the end of the list (and thus self->wr_next == NULL)
then the weakref list itself (and thus the value of *list) will
end up being set to NULL. */
*list = self->wr_next;
self->wr_object = Py_None;
if (self->wr_prev != NULL)
self->wr_prev->wr_next = self->wr_next;
if (self->wr_next != NULL)
self->wr_next->wr_prev = self->wr_prev;
self->wr_prev = NULL;
self->wr_next = NULL;
}
if (callback != NULL) {
Py_DECREF(callback);
self->wr_callback = NULL;
}
}
/* Cyclic gc uses this to *just* clear the passed-in reference, leaving
* the callback intact and uncalled. It must be possible to call self's
* tp_dealloc() after calling this, so self has to be left in a sane enough
* state for that to work. We expect tp_dealloc to decref the callback
* then. The reason for not letting clear_weakref() decref the callback
* right now is that if the callback goes away, that may in turn trigger
* another callback (if a weak reference to the callback exists) -- running
* arbitrary Python code in the middle of gc is a disaster. The convolution
* here allows gc to delay triggering such callbacks until the world is in
* a sane state again.
*/
void
_PyWeakref_ClearRef(PyWeakReference *self)
{
PyObject *callback;
assert(self != NULL);
assert(PyWeakref_Check(self));
/* Preserve and restore the callback around clear_weakref. */
callback = self->wr_callback;
self->wr_callback = NULL;
clear_weakref(self);
self->wr_callback = callback;
}
static void
weakref_dealloc(PyObject *self)
{
PyObject_GC_UnTrack(self);
clear_weakref((PyWeakReference *) self);
Py_TYPE(self)->tp_free(self);
}
static int
gc_traverse(PyWeakReference *self, visitproc visit, void *arg)
{
Py_VISIT(self->wr_callback);
return 0;
}
static int
gc_clear(PyWeakReference *self)
{
clear_weakref(self);
return 0;
}
static PyObject *
weakref_call(PyWeakReference *self, PyObject *args, PyObject *kw)
{
static char *kwlist[] = {NULL};
if (PyArg_ParseTupleAndKeywords(args, kw, ":__call__", kwlist)) {
PyObject *object = PyWeakref_GET_OBJECT(self);
Py_INCREF(object);
return (object);
}
return NULL;
}
static long
weakref_hash(PyWeakReference *self)
{
if (self->hash != -1)
return self->hash;
if (PyWeakref_GET_OBJECT(self) == Py_None) {
PyErr_SetString(PyExc_TypeError, "weak object has gone away");
return -1;
}
self->hash = PyObject_Hash(PyWeakref_GET_OBJECT(self));
return self->hash;
}
static PyObject *
weakref_repr(PyWeakReference *self)
{
char buffer[256];
if (PyWeakref_GET_OBJECT(self) == Py_None) {
PyOS_snprintf(buffer, sizeof(buffer), "<weakref at %p; dead>", self);
}
else {
char *name = NULL;
PyObject *nameobj = PyObject_GetAttrString(PyWeakref_GET_OBJECT(self),
"__name__");
if (nameobj == NULL)
PyErr_Clear();
else if (PyString_Check(nameobj))
name = PyString_AS_STRING(nameobj);
if (name != NULL) {
PyOS_snprintf(buffer, sizeof(buffer),
"<weakref at %p; to '%.50s' at %p (%s)>",
self,
Py_TYPE(PyWeakref_GET_OBJECT(self))->tp_name,
PyWeakref_GET_OBJECT(self),
name);
}
else {
PyOS_snprintf(buffer, sizeof(buffer),
"<weakref at %p; to '%.50s' at %p>",
self,
Py_TYPE(PyWeakref_GET_OBJECT(self))->tp_name,
PyWeakref_GET_OBJECT(self));
}
Py_XDECREF(nameobj);
}
return PyString_FromString(buffer);
}
/* Weak references only support equality, not ordering. Two weak references
are equal if the underlying objects are equal. If the underlying object has
gone away, they are equal if they are identical. */
static PyObject *
weakref_richcompare(PyWeakReference* self, PyWeakReference* other, int op)
{
if ((op != Py_EQ && op != Py_NE) || self->ob_type != other->ob_type) {
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
if (PyWeakref_GET_OBJECT(self) == Py_None
|| PyWeakref_GET_OBJECT(other) == Py_None) {
int res = (self == other);
if (op == Py_NE)
res = !res;
if (res)
Py_RETURN_TRUE;
else
Py_RETURN_FALSE;
}
return PyObject_RichCompare(PyWeakref_GET_OBJECT(self),
PyWeakref_GET_OBJECT(other), op);
}
/* Given the head of an object's list of weak references, extract the
* two callback-less refs (ref and proxy). Used to determine if the
* shared references exist and to determine the back link for newly
* inserted references.
*/
static void
get_basic_refs(PyWeakReference *head,
PyWeakReference **refp, PyWeakReference **proxyp)
{
*refp = NULL;
*proxyp = NULL;
if (head != NULL && head->wr_callback == NULL) {
/* We need to be careful that the "basic refs" aren't
subclasses of the main types. That complicates this a
little. */
if (PyWeakref_CheckRefExact(head)) {
*refp = head;
head = head->wr_next;
}
if (head != NULL
&& head->wr_callback == NULL
&& PyWeakref_CheckProxy(head)) {
*proxyp = head;
/* head = head->wr_next; */
}
}
}
/* Insert 'newref' in the list after 'prev'. Both must be non-NULL. */
static void
insert_after(PyWeakReference *newref, PyWeakReference *prev)
{
newref->wr_prev = prev;
newref->wr_next = prev->wr_next;
if (prev->wr_next != NULL)
prev->wr_next->wr_prev = newref;
prev->wr_next = newref;
}
/* Insert 'newref' at the head of the list; 'list' points to the variable
* that stores the head.
*/
static void
insert_head(PyWeakReference *newref, PyWeakReference **list)
{
PyWeakReference *next = *list;
newref->wr_prev = NULL;
newref->wr_next = next;
if (next != NULL)
next->wr_prev = newref;
*list = newref;
}
static int
parse_weakref_init_args(char *funcname, PyObject *args, PyObject *kwargs,
PyObject **obp, PyObject **callbackp)
{
/* XXX Should check that kwargs == NULL or is empty. */
return PyArg_UnpackTuple(args, funcname, 1, 2, obp, callbackp);
}
static PyObject *
weakref___new__(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
PyWeakReference *self = NULL;
PyObject *ob, *callback = NULL;
if (parse_weakref_init_args("__new__", args, kwargs, &ob, &callback)) {
PyWeakReference *ref, *proxy;
PyWeakReference **list;
if (!PyType_SUPPORTS_WEAKREFS(Py_TYPE(ob))) {
PyErr_Format(PyExc_TypeError,
"cannot create weak reference to '%s' object",
Py_TYPE(ob)->tp_name);
return NULL;
}
if (callback == Py_None)
callback = NULL;
list = GET_WEAKREFS_LISTPTR(ob);
get_basic_refs(*list, &ref, &proxy);
if (callback == NULL && type == &_PyWeakref_RefType) {
if (ref != NULL) {
/* We can re-use an existing reference. */
Py_INCREF(ref);
return (PyObject *)ref;
}
}
/* We have to create a new reference. */
/* Note: the tp_alloc() can trigger cyclic GC, so the weakref
list on ob can be mutated. This means that the ref and
proxy pointers we got back earlier may have been collected,
so we need to compute these values again before we use
them. */
self = (PyWeakReference *) (type->tp_alloc(type, 0));
if (self != NULL) {
init_weakref(self, ob, callback);
if (callback == NULL && type == &_PyWeakref_RefType) {
insert_head(self, list);
}
else {
PyWeakReference *prev;
get_basic_refs(*list, &ref, &proxy);
prev = (proxy == NULL) ? ref : proxy;
if (prev == NULL)
insert_head(self, list);
else
insert_after(self, prev);
}
}
}
return (PyObject *)self;
}
static int
weakref___init__(PyObject *self, PyObject *args, PyObject *kwargs)
{
PyObject *tmp;
if (parse_weakref_init_args("__init__", args, kwargs, &tmp, &tmp))
return 0;
else
return -1;
}
PyTypeObject
_PyWeakref_RefType = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"weakref",
sizeof(PyWeakReference),
0,
weakref_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
(reprfunc)weakref_repr, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
(hashfunc)weakref_hash, /*tp_hash*/
(ternaryfunc)weakref_call, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_HAVE_RICHCOMPARE
| Py_TPFLAGS_BASETYPE, /*tp_flags*/
0, /*tp_doc*/
(traverseproc)gc_traverse, /*tp_traverse*/
(inquiry)gc_clear, /*tp_clear*/
(richcmpfunc)weakref_richcompare, /*tp_richcompare*/
0, /*tp_weaklistoffset*/
0, /*tp_iter*/
0, /*tp_iternext*/
0, /*tp_methods*/
0, /*tp_members*/
0, /*tp_getset*/
0, /*tp_base*/
0, /*tp_dict*/
0, /*tp_descr_get*/
0, /*tp_descr_set*/
0, /*tp_dictoffset*/
weakref___init__, /*tp_init*/
PyType_GenericAlloc, /*tp_alloc*/
weakref___new__, /*tp_new*/
PyObject_GC_Del, /*tp_free*/
};
static int
proxy_checkref(PyWeakReference *proxy)
{
if (PyWeakref_GET_OBJECT(proxy) == Py_None) {
PyErr_SetString(PyExc_ReferenceError,
"weakly-referenced object no longer exists");
return 0;
}
return 1;
}
/* If a parameter is a proxy, check that it is still "live" and wrap it,
* replacing the original value with the raw object. Raises ReferenceError
* if the param is a dead proxy.
*/
#define UNWRAP(o) \
if (PyWeakref_CheckProxy(o)) { \
if (!proxy_checkref((PyWeakReference *)o)) \
return NULL; \
o = PyWeakref_GET_OBJECT(o); \
}
#define UNWRAP_I(o) \
if (PyWeakref_CheckProxy(o)) { \
if (!proxy_checkref((PyWeakReference *)o)) \
return -1; \
o = PyWeakref_GET_OBJECT(o); \
}
#define WRAP_UNARY(method, generic) \
static PyObject * \
method(PyObject *proxy) { \
UNWRAP(proxy); \
return generic(proxy); \
}
#define WRAP_BINARY(method, generic) \
static PyObject * \
method(PyObject *x, PyObject *y) { \
UNWRAP(x); \
UNWRAP(y); \
return generic(x, y); \
}
/* Note that the third arg needs to be checked for NULL since the tp_call
* slot can receive NULL for this arg.
*/
#define WRAP_TERNARY(method, generic) \
static PyObject * \
method(PyObject *proxy, PyObject *v, PyObject *w) { \
UNWRAP(proxy); \
UNWRAP(v); \
if (w != NULL) \
UNWRAP(w); \
return generic(proxy, v, w); \
}
#define WRAP_METHOD(method, special) \
static PyObject * \
method(PyObject *proxy) { \
UNWRAP(proxy); \
return PyObject_CallMethod(proxy, special, ""); \
}
/* direct slots */
WRAP_BINARY(proxy_getattr, PyObject_GetAttr)
WRAP_UNARY(proxy_str, PyObject_Str)
WRAP_TERNARY(proxy_call, PyEval_CallObjectWithKeywords)
static PyObject *
proxy_repr(PyWeakReference *proxy)
{
char buf[160];
PyOS_snprintf(buf, sizeof(buf),
"<weakproxy at %p to %.100s at %p>", proxy,
Py_TYPE(PyWeakref_GET_OBJECT(proxy))->tp_name,
PyWeakref_GET_OBJECT(proxy));
return PyString_FromString(buf);
}
static int
proxy_setattr(PyWeakReference *proxy, PyObject *name, PyObject *value)
{
if (!proxy_checkref(proxy))
return -1;
return PyObject_SetAttr(PyWeakref_GET_OBJECT(proxy), name, value);
}
static int
proxy_compare(PyObject *proxy, PyObject *v)
{
UNWRAP_I(proxy);
UNWRAP_I(v);
return PyObject_Compare(proxy, v);
}
/* number slots */
WRAP_BINARY(proxy_add, PyNumber_Add)
WRAP_BINARY(proxy_sub, PyNumber_Subtract)
WRAP_BINARY(proxy_mul, PyNumber_Multiply)
WRAP_BINARY(proxy_div, PyNumber_Divide)
WRAP_BINARY(proxy_floor_div, PyNumber_FloorDivide)
WRAP_BINARY(proxy_true_div, PyNumber_TrueDivide)
WRAP_BINARY(proxy_mod, PyNumber_Remainder)
WRAP_BINARY(proxy_divmod, PyNumber_Divmod)
WRAP_TERNARY(proxy_pow, PyNumber_Power)
WRAP_UNARY(proxy_neg, PyNumber_Negative)
WRAP_UNARY(proxy_pos, PyNumber_Positive)
WRAP_UNARY(proxy_abs, PyNumber_Absolute)
WRAP_UNARY(proxy_invert, PyNumber_Invert)
WRAP_BINARY(proxy_lshift, PyNumber_Lshift)
WRAP_BINARY(proxy_rshift, PyNumber_Rshift)
WRAP_BINARY(proxy_and, PyNumber_And)
WRAP_BINARY(proxy_xor, PyNumber_Xor)
WRAP_BINARY(proxy_or, PyNumber_Or)
WRAP_UNARY(proxy_int, PyNumber_Int)
WRAP_UNARY(proxy_long, PyNumber_Long)
WRAP_UNARY(proxy_float, PyNumber_Float)
WRAP_BINARY(proxy_iadd, PyNumber_InPlaceAdd)
WRAP_BINARY(proxy_isub, PyNumber_InPlaceSubtract)
WRAP_BINARY(proxy_imul, PyNumber_InPlaceMultiply)
WRAP_BINARY(proxy_idiv, PyNumber_InPlaceDivide)
WRAP_BINARY(proxy_ifloor_div, PyNumber_InPlaceFloorDivide)
WRAP_BINARY(proxy_itrue_div, PyNumber_InPlaceTrueDivide)
WRAP_BINARY(proxy_imod, PyNumber_InPlaceRemainder)
WRAP_TERNARY(proxy_ipow, PyNumber_InPlacePower)
WRAP_BINARY(proxy_ilshift, PyNumber_InPlaceLshift)
WRAP_BINARY(proxy_irshift, PyNumber_InPlaceRshift)
WRAP_BINARY(proxy_iand, PyNumber_InPlaceAnd)
WRAP_BINARY(proxy_ixor, PyNumber_InPlaceXor)
WRAP_BINARY(proxy_ior, PyNumber_InPlaceOr)
WRAP_UNARY(proxy_index, PyNumber_Index)
static int
proxy_nonzero(PyWeakReference *proxy)
{
PyObject *o = PyWeakref_GET_OBJECT(proxy);
if (!proxy_checkref(proxy))
return -1;
return PyObject_IsTrue(o);
}
static void
proxy_dealloc(PyWeakReference *self)
{
if (self->wr_callback != NULL)
PyObject_GC_UnTrack((PyObject *)self);
clear_weakref(self);
PyObject_GC_Del(self);
}
/* sequence slots */
static PyObject *
proxy_slice(PyWeakReference *proxy, Py_ssize_t i, Py_ssize_t j)
{
if (!proxy_checkref(proxy))
return NULL;
return PySequence_GetSlice(PyWeakref_GET_OBJECT(proxy), i, j);
}
static int
proxy_ass_slice(PyWeakReference *proxy, Py_ssize_t i, Py_ssize_t j, PyObject *value)
{
if (!proxy_checkref(proxy))
return -1;
return PySequence_SetSlice(PyWeakref_GET_OBJECT(proxy), i, j, value);
}
static int
proxy_contains(PyWeakReference *proxy, PyObject *value)
{
if (!proxy_checkref(proxy))
return -1;
return PySequence_Contains(PyWeakref_GET_OBJECT(proxy), value);
}
/* mapping slots */
static Py_ssize_t
proxy_length(PyWeakReference *proxy)
{
if (!proxy_checkref(proxy))
return -1;
return PyObject_Length(PyWeakref_GET_OBJECT(proxy));
}
WRAP_BINARY(proxy_getitem, PyObject_GetItem)
static int
proxy_setitem(PyWeakReference *proxy, PyObject *key, PyObject *value)
{
if (!proxy_checkref(proxy))
return -1;
if (value == NULL)
return PyObject_DelItem(PyWeakref_GET_OBJECT(proxy), key);
else
return PyObject_SetItem(PyWeakref_GET_OBJECT(proxy), key, value);
}
/* iterator slots */
static PyObject *
proxy_iter(PyWeakReference *proxy)
{
if (!proxy_checkref(proxy))
return NULL;
return PyObject_GetIter(PyWeakref_GET_OBJECT(proxy));
}
static PyObject *
proxy_iternext(PyWeakReference *proxy)
{
if (!proxy_checkref(proxy))
return NULL;
return PyIter_Next(PyWeakref_GET_OBJECT(proxy));
}
WRAP_METHOD(proxy_unicode, "__unicode__");
static PyMethodDef proxy_methods[] = {
{"__unicode__", (PyCFunction)proxy_unicode, METH_NOARGS},
{NULL, NULL}
};
static PyNumberMethods proxy_as_number = {
proxy_add, /*nb_add*/
proxy_sub, /*nb_subtract*/
proxy_mul, /*nb_multiply*/
proxy_div, /*nb_divide*/
proxy_mod, /*nb_remainder*/
proxy_divmod, /*nb_divmod*/
proxy_pow, /*nb_power*/
proxy_neg, /*nb_negative*/
proxy_pos, /*nb_positive*/
proxy_abs, /*nb_absolute*/
(inquiry)proxy_nonzero, /*nb_nonzero*/
proxy_invert, /*nb_invert*/
proxy_lshift, /*nb_lshift*/
proxy_rshift, /*nb_rshift*/
proxy_and, /*nb_and*/
proxy_xor, /*nb_xor*/
proxy_or, /*nb_or*/
0, /*nb_coerce*/
proxy_int, /*nb_int*/
proxy_long, /*nb_long*/
proxy_float, /*nb_float*/
0, /*nb_oct*/
0, /*nb_hex*/
proxy_iadd, /*nb_inplace_add*/
proxy_isub, /*nb_inplace_subtract*/
proxy_imul, /*nb_inplace_multiply*/
proxy_idiv, /*nb_inplace_divide*/
proxy_imod, /*nb_inplace_remainder*/
proxy_ipow, /*nb_inplace_power*/
proxy_ilshift, /*nb_inplace_lshift*/
proxy_irshift, /*nb_inplace_rshift*/
proxy_iand, /*nb_inplace_and*/
proxy_ixor, /*nb_inplace_xor*/
proxy_ior, /*nb_inplace_or*/
proxy_floor_div, /*nb_floor_divide*/
proxy_true_div, /*nb_true_divide*/
proxy_ifloor_div, /*nb_inplace_floor_divide*/
proxy_itrue_div, /*nb_inplace_true_divide*/
proxy_index, /*nb_index*/
};
static PySequenceMethods proxy_as_sequence = {
(lenfunc)proxy_length, /*sq_length*/
0, /*sq_concat*/
0, /*sq_repeat*/
0, /*sq_item*/
(ssizessizeargfunc)proxy_slice, /*sq_slice*/
0, /*sq_ass_item*/
(ssizessizeobjargproc)proxy_ass_slice, /*sq_ass_slice*/
(objobjproc)proxy_contains, /* sq_contains */
};
static PyMappingMethods proxy_as_mapping = {
(lenfunc)proxy_length, /*mp_length*/
proxy_getitem, /*mp_subscript*/
(objobjargproc)proxy_setitem, /*mp_ass_subscript*/
};
PyTypeObject
_PyWeakref_ProxyType = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"weakproxy",
sizeof(PyWeakReference),
0,
/* methods */
(destructor)proxy_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
proxy_compare, /* tp_compare */
(reprfunc)proxy_repr, /* tp_repr */
&proxy_as_number, /* tp_as_number */
&proxy_as_sequence, /* tp_as_sequence */
&proxy_as_mapping, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
proxy_str, /* tp_str */
proxy_getattr, /* tp_getattro */
(setattrofunc)proxy_setattr, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC
| Py_TPFLAGS_CHECKTYPES, /* tp_flags */
0, /* tp_doc */
(traverseproc)gc_traverse, /* tp_traverse */
(inquiry)gc_clear, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
(getiterfunc)proxy_iter, /* tp_iter */
(iternextfunc)proxy_iternext, /* tp_iternext */
proxy_methods, /* tp_methods */
};
PyTypeObject
_PyWeakref_CallableProxyType = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"weakcallableproxy",
sizeof(PyWeakReference),
0,
/* methods */
(destructor)proxy_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
proxy_compare, /* tp_compare */
(unaryfunc)proxy_repr, /* tp_repr */
&proxy_as_number, /* tp_as_number */
&proxy_as_sequence, /* tp_as_sequence */
&proxy_as_mapping, /* tp_as_mapping */
0, /* tp_hash */
proxy_call, /* tp_call */
proxy_str, /* tp_str */
proxy_getattr, /* tp_getattro */
(setattrofunc)proxy_setattr, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC
| Py_TPFLAGS_CHECKTYPES, /* tp_flags */
0, /* tp_doc */
(traverseproc)gc_traverse, /* tp_traverse */
(inquiry)gc_clear, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
(getiterfunc)proxy_iter, /* tp_iter */
(iternextfunc)proxy_iternext, /* tp_iternext */
};
PyObject *
PyWeakref_NewRef(PyObject *ob, PyObject *callback)
{
PyWeakReference *result = NULL;
PyWeakReference **list;
PyWeakReference *ref, *proxy;
if (!PyType_SUPPORTS_WEAKREFS(Py_TYPE(ob))) {
PyErr_Format(PyExc_TypeError,
"cannot create weak reference to '%s' object",
Py_TYPE(ob)->tp_name);
return NULL;
}
list = GET_WEAKREFS_LISTPTR(ob);
get_basic_refs(*list, &ref, &proxy);
if (callback == Py_None)
callback = NULL;
if (callback == NULL)
/* return existing weak reference if it exists */
result = ref;
if (result != NULL)
Py_INCREF(result);
else {
/* Note: new_weakref() can trigger cyclic GC, so the weakref
list on ob can be mutated. This means that the ref and
proxy pointers we got back earlier may have been collected,
so we need to compute these values again before we use
them. */
result = new_weakref(ob, callback);
if (result != NULL) {
get_basic_refs(*list, &ref, &proxy);
if (callback == NULL) {
if (ref == NULL)
insert_head(result, list);
else {
/* Someone else added a ref without a callback
during GC. Return that one instead of this one
to avoid violating the invariants of the list
of weakrefs for ob. */
Py_DECREF(result);
Py_INCREF(ref);
result = ref;
}
}
else {
PyWeakReference *prev;
prev = (proxy == NULL) ? ref : proxy;
if (prev == NULL)
insert_head(result, list);
else
insert_after(result, prev);
}
}
}
return (PyObject *) result;
}
PyObject *
PyWeakref_NewProxy(PyObject *ob, PyObject *callback)
{
PyWeakReference *result = NULL;
PyWeakReference **list;
PyWeakReference *ref, *proxy;
if (!PyType_SUPPORTS_WEAKREFS(Py_TYPE(ob))) {
PyErr_Format(PyExc_TypeError,
"cannot create weak reference to '%s' object",
Py_TYPE(ob)->tp_name);
return NULL;
}
list = GET_WEAKREFS_LISTPTR(ob);
get_basic_refs(*list, &ref, &proxy);
if (callback == Py_None)
callback = NULL;
if (callback == NULL)
/* attempt to return an existing weak reference if it exists */
result = proxy;
if (result != NULL)
Py_INCREF(result);
else {
/* Note: new_weakref() can trigger cyclic GC, so the weakref
list on ob can be mutated. This means that the ref and
proxy pointers we got back earlier may have been collected,
so we need to compute these values again before we use
them. */
result = new_weakref(ob, callback);
if (result != NULL) {
PyWeakReference *prev;
if (PyCallable_Check(ob))
Py_TYPE(result) = &_PyWeakref_CallableProxyType;
else
Py_TYPE(result) = &_PyWeakref_ProxyType;
get_basic_refs(*list, &ref, &proxy);
if (callback == NULL) {
if (proxy != NULL) {
/* Someone else added a proxy without a callback
during GC. Return that one instead of this one
to avoid violating the invariants of the list
of weakrefs for ob. */
Py_DECREF(result);
Py_INCREF(result = proxy);
goto skip_insert;
}
prev = ref;
}
else
prev = (proxy == NULL) ? ref : proxy;
if (prev == NULL)
insert_head(result, list);
else
insert_after(result, prev);
skip_insert:
;
}
}
return (PyObject *) result;
}
PyObject *
PyWeakref_GetObject(PyObject *ref)
{
if (ref == NULL || !PyWeakref_Check(ref)) {
PyErr_BadInternalCall();
return NULL;
}
return PyWeakref_GET_OBJECT(ref);
}
/* Note that there's an inlined copy-paste of handle_callback() in gcmodule.c's
* handle_weakrefs().
*/
static void
handle_callback(PyWeakReference *ref, PyObject *callback)
{
PyObject *cbresult = PyObject_CallFunctionObjArgs(callback, ref, NULL);
if (cbresult == NULL)
PyErr_WriteUnraisable(callback);
else
Py_DECREF(cbresult);
}
/* This function is called by the tp_dealloc handler to clear weak references.
*
* This iterates through the weak references for 'object' and calls callbacks
* for those references which have one. It returns when all callbacks have
* been attempted.
*/
void
PyObject_ClearWeakRefs(PyObject *object)
{
PyWeakReference **list;
if (object == NULL
|| !PyType_SUPPORTS_WEAKREFS(Py_TYPE(object))
|| object->ob_refcnt != 0) {
PyErr_BadInternalCall();
return;
}
list = GET_WEAKREFS_LISTPTR(object);
/* Remove the callback-less basic and proxy references */
if (*list != NULL && (*list)->wr_callback == NULL) {
clear_weakref(*list);
if (*list != NULL && (*list)->wr_callback == NULL)
clear_weakref(*list);
}
if (*list != NULL) {
PyWeakReference *current = *list;
Py_ssize_t count = _PyWeakref_GetWeakrefCount(current);
PyObject *err_type, *err_value, *err_tb;
PyErr_Fetch(&err_type, &err_value, &err_tb);
if (count == 1) {
PyObject *callback = current->wr_callback;
current->wr_callback = NULL;
clear_weakref(current);
if (callback != NULL) {
if (current->ob_refcnt > 0)
handle_callback(current, callback);
Py_DECREF(callback);
}
}
else {
PyObject *tuple;
Py_ssize_t i = 0;
tuple = PyTuple_New(count * 2);
if (tuple == NULL) {
_PyErr_ReplaceException(err_type, err_value, err_tb);
return;
}
for (i = 0; i < count; ++i) {
PyWeakReference *next = current->wr_next;
if (current->ob_refcnt > 0)
{
Py_INCREF(current);
PyTuple_SET_ITEM(tuple, i * 2, (PyObject *) current);
PyTuple_SET_ITEM(tuple, i * 2 + 1, current->wr_callback);
}
else {
Py_DECREF(current->wr_callback);
}
current->wr_callback = NULL;
clear_weakref(current);
current = next;
}
for (i = 0; i < count; ++i) {
PyObject *callback = PyTuple_GET_ITEM(tuple, i * 2 + 1);
/* The tuple may have slots left to NULL */
if (callback != NULL) {
PyObject *item = PyTuple_GET_ITEM(tuple, i * 2);
handle_callback((PyWeakReference *)item, callback);
}
}
Py_DECREF(tuple);
}
assert(!PyErr_Occurred());
PyErr_Restore(err_type, err_value, err_tb);
}
}

File diff suppressed because it is too large Load Diff