<previous | contents | next> Pyro Manual

11. Pyro's Implementation

The purpose of this chapter is to provide some insight in the Pyro's implementation. However please keep in mind that it is quite tedious for me to update this chapter with each Pyro release, so I want to stress the fact that the source code is probably the best place to look if you want to find out specific details about Pyro's implementation.

Nevertheless this chapter is a nice starting point to explore the different parts that make up Pyro.

The Pyro Principle

How does Pyro work??? Pyro uses two main principles to do what it does: method call interception (actually, attribute access interception) and marshalling.

Your code doesn't call a Pyro object directly. It calls a method on a proxy that acts just like the Pyro object it represents. The proxy intercepts the method call by using a special implementation of __getattr__, that passes the call to _invokePYRO. In turn, that method forwards the call to the protocol adapter, that creates a message containing all the details of the call (using pickle to marshall all Python objects). The message is passed via the network to the Pyro daemon on the other side.

The daemon receives the message and unmarshalls the request. It now knows the objectID of the required Pyro objects, the desired member function to call, and its arguments. It looks up the Pyro Object in its table and calls the Pyro_dyncall method. This method is in ObjBase which is the base class of all Pyro objects. This method uses the appropriate apply call on itself to call the actual method. The result of the method is handed back to the Pyro daemon, that marshalls it, and passes it back over the network to the calling protocol adapter. The adapter on the client side unmarshalls the result data, hands it back to the proxy that called it, and then the _invokePYRO method of the proxy returns the result data as if it was obtained using a regular method call on a local object!

This was Pyro in a nutshell. There are many more things to consider, such as transporting modules (mobile objects or even mobile agents) and oneway calls, but you have to read the source anyway to find all the gory details.

The command-line tools (bin/*)

The command-line tools are all in fact very small Python scripts. They contain a few Python statements that start the actual tools, which are part of the Pyro package:
Tool Implemented in Notes
es Pyro.EventService.Server start Event Server
nsc Pyro.nsc control Name Server
xnsc Pyro.xnsc Tkinter-GUI version of nsc
wxnsc Pyro.wxnsc WxPython/WxWindows version of nsc, with nice tree view of namespace
genguid Pyro.util create a GUID, entrypoint is genguid_scripthelper()
ns / rns Pyro.naming start Name Server, entrypoint is main()
nsd / esd Pyro.ext.daemonizer Unix initd daemon scripts for name server and event server
nssvc / essvc Pyro.ext.NS_NtService and ES_NTService Windows NT Service control scripts for NS and ES

Pyro package initializer (Pyro/__init__.py)

Pyro is a Python package and as such is contained in the 'Pyro' directory. This directory contains the different module files that make up the Pyro system. One of them is the initialization file, __init__.py. It loads the configuration settings for Pyro (Pyro.config.* items). This way, whenever a module from the Pyro package is imported, it can use the config items right away.

Within the core package is also another package; Pyro.EventService. The event service is implemented here: the Event Server itself and the base classes for event publishers and listeners.

Configuration (Pyro/configuration.py)

This module contains the logic that deals with Pyro's configuration. There is a Config class and a ConfigReader class. The Config class is the container for all configuration items. The instance is available as Pyro.config (created in the package initializer, see above). The configuration items are all attributes of this object. Config uses the ConfigReader to read Pyro's configuration file. It deals with defaults and environment settings too.

Constants (Pyro/constants.py)

This module contains all globally used constants. By placing them in a distinct module that doesn't import any other modules (apart from util), we can also prevent import cycles between other Pyro modules. Things that are defined here are for example the Pyro version string, and the names for the Name Server and Event Server.

Core library (Pyro/core.py)

This module contains all core logic of Pyro that is not tied to the network protocol that is used. This means that you will not find any TCP/IP socket specific code in the core module; the protocol module contains this. What the core module does contain is all stuff that realize the core functions of Pyro:
ObjBase
Server-side object implementation base class or master class with the actual object as delegate. It supplies the methods for remote method invocation (Pyro_dyncall) and remote attribute access (remote_getattr etc.)
PyroURI
Pyro Universal Resource Identifier. This class represents a Pyro URI (which consists of four parts: a protocol identifier, an IP address, a portnumber, and an object ID). PyroURI can be converted to and from a - human readable - string.
DynamicProxy and DynamicProxyWithAttrs
These two classes are the dynamic Pyro proxies. They can be used by clients to invoke objects for which they have no precompiled proxy. The "WithAttrs" proxy is needed if you want to do direct attribute access on remote Pyro objects. The proxies have a special _invokePYRO method that intercepts method calls to pass them via the protocol adapter to the remote Pyro object. Special care is taken to make proxy objects suitable for pickling so they can be transported trough the Pyro protocol across the network.
getProxyForURI and getAttrProxyForURI
Helper functions to create proxies directly from a given Pyro URI (object).
Daemon
The server-side Pyro daemon. Accepts and dispatches incoming Pyro method calls. It is derived from the low level TCPServer and contains the server end of a protocol adapter. The daemon passes incoming method calls (via its handleRequest method) to the protocol adapter that sorts out what to do exactly. The connect and disconnect methods are used to keep track of server side Pyro objects. The Daemon's handleError is used in case of a Pyro error, it processes the error and cleans things up.
Pyro client and Pyro server initialization code
The initClient and initServer functions perform necessary initialization. For instance, the server init code first checks the availability of the PYRO_STORAGE directory.

Name Server (Pyro/naming.py)

This module contains all logic that deals with object naming and locations. Pyro's Name Server is implemented here as well as various other things.
NameServerLocator
Used to locate the Name Server by using various lookup methods (broadcast, direct lookup). Usually you call the getNS method and you get a Pyro Proxy for the Name Server back.
NameServerProxy
A built-in proxy for the Name Server that takes care of the naming rules (global names, default groups). You get this back from the NameServerLocator.getNS.
NameServer and PersistentNameServer
The actual pyro object that is the Name Server. It is created by some utility code that also connects it to a daemon. This code is called by the ns script that starts the Name Server.
Various classes for hierarchical naming support, they implement a tree structure.
We have a (name,value) tuple NameValue and a NamedTree recursive data structure.
BroadcastServer
The helper server that listens to UDP broadcast events from the locator, and replies with the URI of the Name Server object. This server's socket is joined with the regular Pyro Daemon that listens for incoming Pyro calls for the NS object.

Protocol Adapters (Pyro/protocol.py)

The idea behind this module is to make it fairly straightforward to switch the underlying protocol that Pyro uses. Currently only one implementation is available: the native PYRO protocol that sits on top of TCP/IP. This module contains all TCP/IP socket related code.

Utilities (Pyro/util.py)

This module defines various utility functions and classes. The most important ones are:
LoggerBase
This is the abstract base class of the various logging classes, see the next two items. Don't use this directly.
SystemLogger
An object of this class is directly available as Pyro.util.Log and this is Pyro's system logger that writes to Pyro's system logfile.
UserLogger
An object of this class can be constructed to create a user logger that writes to the user logfile.
ArgParser
A simple command line argument parser like the getopt function in ANSI C. Pyro's command line tools use this. It can parse command line options according to an option specification string. The getOpt member is used to read the results.
getGUID
Generates a new GUID. This is essentially a big number that is unique in time and in space. Unique in time means that every moment you generate a new GUID it is different from the ones you generated before. Pyro does this by using a time stamp as part of the GUID. Unique in space means that when you generate a GUID on a different machine, it will be different from the GUID you generated elsewhere (even when you generate them exactly the same moment). Pyro does this by using the network address of your computer as part of the GUID, and the Python process id. The latter is necessary because multiple Python processes may be running on the same IP address. The 128-bit GUID is returned as a hexlified string of 32 characters.
genguid_scripthelper
This helper function is used by the genguid command line utility, which prints a new GUID.
getPyroTraceback
Gets the remote traceback from the exception that is caught on the client, augments this with the local traceback, and returns a list of traceback steps. Use this to find out which part of the remote code caused the exception.

Exception definitions (Pyro/errors.py)

This module defines Pyro.PyroError and its derived exceptions. PyroError is the Pyro exception type that is used for problems within Pyro. User code should not use it! Also, this module defines the PyroExceptionCapsule class, which is used to represent any Python exception that has to be transported across the network, and raised on the other side (by invoking raiseEx on this object).

Event Server package (Pyro/EventService)

This package contains the Event Server modules:
Pyro/EventService/Clients.py
Here are the Subscriber and Publisher base classes defined that you can use to easily build publisher or listener clients for the ES.
Pyro/EventService/Server.py
Contains the actual EventService Pyro object that is started by some utility code. This module is called by the es script that you can use to start the Event Server.

Extensions package (Pyro/ext)

This package contains optional modules that provide certain extensions to the core Pyro library. It contains:
Pyro/ext/remote.py
The easy-remoting module that is shown in the "quickstart" example.
Pyro/ext/remote_nons.py
The easy-remoting module that doesn't use the Name Server, it is shown in the "quickstart-noNS" example.
Pyro/ext/daemonizer.py
Utility classes to "daemonize" a Pyro server (Unix only). Used by the nsd and esd initd daemon scripts for instance.
Pyro/ext/BasicNTService.py
Utility classes for creating Windows NT Services (Windows NT only).
Pyro/ext/NS_NtService.py
The Name Server as Windows NT Service.
Pyro/ext/ES_NtService.py
The Event Server as Windows NT Service.
Pyro/ext/ServiceTest.py
Test sets for the Windows NT Services.
<previous | contents | next> Pyro Manual