Callbacks¶
For more fine-grained control, libcurl allows a number of callbacks to be
associated with each connection. In pycurl, callbacks are defined using the
setopt() method for Curl objects with options WRITEFUNCTION,
READFUNCTION, HEADERFUNCTION, PROGRESSFUNCTION,
XFERINFOFUNCTION, IOCTLFUNCTION, DEBUGFUNCTION,
PREREQFUNCTION, TRAILERFUNCTION, RESOLVER_START_FUNCTION,
FNMATCH_FUNCTION, HSTSREADFUNCTION or HSTSWRITEFUNCTION.
These options correspond to the libcurl options with CURLOPT_
prefix removed. A callback in pycurl must be either a regular Python
function, a class method or an extension type function.
There are some limitations to some of the options which can be used
concurrently with the pycurl callbacks compared to the libcurl callbacks.
This is to allow different callback functions to be associated with different
Curl objects. More specifically, WRITEDATA cannot be used with
WRITEFUNCTION, READDATA cannot be used with READFUNCTION,
WRITEHEADER cannot be used with HEADERFUNCTION.
In practice, these limitations can be overcome by having a
callback function be a class instance method and rather use the class
instance attributes to store per object data such as files used in the
callbacks.
The signature of each callback used in PycURL is documented below.
Error Reporting¶
PycURL callbacks are invoked as follows:
Python application -> perform() -> libcurl (C code) -> Python callback
Because callbacks are invoked by libcurl, they should not raise exceptions on failure but instead return appropriate values indicating failure. The documentation for individual callbacks below specifies expected success and failure return values.
Unhandled exceptions propagated out of Python callbacks will be intercepted
by PycURL or the Python runtime. This will fail the callback with a
generic failure status, in turn failing the perform() operation.
A failing perform() will raise pycurl.error, but the error code
used depends on the specific callback.
KeyboardInterrupt and other BaseException subclasses (for example, SystemExit)
are handled specially: if they are raised inside a callback, they are preserved and re-raised
to the caller instead of being converted into a pycurl.error.
Callbacks that libcurl invokes while a handle is being closed have no
perform() to fail, whether the close came from close() or from the last
reference going away. Exceptions raised there are written to
sys.unraisablehook.
Rich context information like exception objects can be stored in various ways, for example the following example stores OPENSOCKET callback exception on the Curl object:
import pycurl, random, socket
class ConnectionRejected(Exception):
pass
def opensocket(curl, purpose, curl_address):
# always fail
curl.exception = ConnectionRejected('Rejecting connection attempt in opensocket callback')
return pycurl.SOCKET_BAD
# the callback must create a socket if it does not fail,
# see examples/opensocketexception.py
c = pycurl.Curl()
c.setopt(c.URL, 'https://pycurl.github.io')
c.exception = None
c.setopt(c.OPENSOCKETFUNCTION,
lambda purpose, address: opensocket(c, purpose, address))
try:
c.perform()
except pycurl.error as e:
if e.args[0] == pycurl.E_COULDNT_CONNECT and c.exception:
print(c.exception)
else:
print(e)
WRITEFUNCTION¶
- WRITEFUNCTION(byte string) number of characters written¶
Callback for writing data. Corresponds to CURLOPT_WRITEFUNCTION in libcurl.
On Python 3, the argument is of type
bytes.The
WRITEFUNCTIONcallback may return the number of bytes written. If this number is not equal to the size of the byte string, this signifies an error and libcurl will abort the request. ReturningNoneis an alternate way of indicating that the callback has consumed all of the string passed to it and, hence, succeeded.Passing
use_memoryview=TruetoCurl.setoptcauses the callback to receive a read-onlymemoryviewover libcurl’s temporary buffer instead of abytescopy. The memoryview is released as soon as the callback returns, so any reference kept past that point raisesValueErroron access. Views derived from it, such as slices ormemoryview()wrappers, are not released with it and give no such protection: using one after the callback returns reads freed memory. Copy what you need to keep withbytes(chunk).Example:
def write_cb(chunk: memoryview) -> int: return len(chunk) curl.setopt(pycurl.WRITEFUNCTION, write_cb, use_memoryview=True)
write_test.py test shows how to use
WRITEFUNCTION.
Example: Callbacks for document header and body¶
This example prints the header data to stderr and the body data to stdout. Also note that neither callback returns the number of bytes written. For WRITEFUNCTION and HEADERFUNCTION callbacks, returning None implies that all bytes where written.
## Callback function invoked when body data is ready
def body(buf):
# Print body data to stdout
import sys
sys.stdout.write(buf)
# Returning None implies that all bytes were written
## Callback function invoked when header data is ready
def header(buf):
# Print header data to stderr
import sys
sys.stderr.write(buf)
# Returning None implies that all bytes were written
c = pycurl.Curl()
c.setopt(pycurl.URL, "http://www.python.org/")
c.setopt(pycurl.WRITEFUNCTION, body)
c.setopt(pycurl.HEADERFUNCTION, header)
c.perform()
HEADERFUNCTION¶
- HEADERFUNCTION(byte string) number of characters written¶
Callback for writing received headers. Corresponds to CURLOPT_HEADERFUNCTION in libcurl.
On Python 3, the argument is of type
bytes.The
HEADERFUNCTIONcallback may return the number of bytes written. If this number is not equal to the size of the byte string, this signifies an error and libcurl will abort the request. ReturningNoneis an alternate way of indicating that the callback has consumed all of the string passed to it and, hence, succeeded.Passing
use_memoryview=TruetoCurl.setoptcauses the callback to receive a read-onlymemoryviewover libcurl’s temporary buffer instead of abytescopy. The memoryview is released as soon as the callback returns, so any reference kept past that point raisesValueErroron access. Views derived from it, such as slices ormemoryview()wrappers, are not released with it and give no such protection: using one after the callback returns reads freed memory. Copy what you need to keep withbytes(chunk).Example:
def header_cb(chunk: memoryview) -> int: return len(chunk) curl.setopt(pycurl.HEADERFUNCTION, header_cb, use_memoryview=True)
header_test.py test shows how to use
WRITEFUNCTION.
READFUNCTION¶
- READFUNCTION(number of characters to read) byte string¶
Callback for reading data. Corresponds to CURLOPT_READFUNCTION in libcurl.
The callback must return either a C-contiguous object supporting the buffer protocol (e.g. bytes, bytearray, memoryview, or a C-contiguous numpy array) or a Unicode string consisting of ASCII code points only.
In addition,
READFUNCTIONmay returnREADFUNC_ABORTorREADFUNC_PAUSE. See the libcurl documentation for an explanation of these values.The file_upload.py example in the distribution contains example code for using
READFUNCTION.
SEEKFUNCTION¶
- SEEKFUNCTION(offset, origin) status¶
Callback for seek operations. Corresponds to CURLOPT_SEEKFUNCTION in libcurl.
originis0(beginning),1(current position) or2(end).Return one of:
SEEKFUNC_OK(seek succeeded)SEEKFUNC_FAIL(hard failure)SEEKFUNC_CANTSEEK(seek not possible; libcurl may fall back)
IOCTLFUNCTION¶
- IOCTLFUNCTION(ioctl cmd) status¶
Callback for I/O operations. Corresponds to CURLOPT_IOCTLFUNCTION in libcurl.
Note: this callback is deprecated. Use SEEKFUNCTION instead.
DEBUGFUNCTION¶
- DEBUGFUNCTION(debug message type, debug message byte string) None¶
Callback for debug information. Corresponds to CURLOPT_DEBUGFUNCTION in libcurl.
Changed in version 7.19.5.2: The second argument to a
DEBUGFUNCTIONcallback is now of typebyteson Python 3. Previously the argument was of typestr.debug_test.py test shows how to use
DEBUGFUNCTION.
Example: Debug callbacks¶
This example shows how to use the debug callback. The debug message type is an integer indicating the type of debug message. The VERBOSE option must be enabled for this callback to be invoked.
def test(debug_type, debug_msg):
print