| <previous | contents | next> | Pyro Manual |
print RemoteObj.name prints the attribute 'name' of the remote object 'RemoteObj').
There are a few important points to keep in mind when you're using this:
RemoteObj.person.address.street) or 'nested'
method calls (such as RemoteObj.person.address.getStreet()):
the rule counts for each attribute.
When you call object.sub1.sub2 where object is a Pyro proxy to your
Pyro server object, Python processes the statement left-to-right.
First Python wants to get the sub1 attribute from object. So Pyro
intercepts a getattr call from Python and the remote Pyro object
returns the sub1 attribute. Unfortunately, this value is an object
(i.e. a class instance)! The client receives a class instance of a class
it doesn't know about! (that's the reason it crashes with
AttributeError: 'module' object has no attribute 'SomeClass', when the class
of the attribute object is not available on the client).
The easiest way to solve this is to add a delegation method to
your Pyro object (something like getSub2() that returns self.sub1.sub2.
Then, don't access the object.sub1.sub2 attribute directly, but get it with object.getSub2().
The other solution is to make the class available to the client process:
put it in a separate module and place the module somewhere
the client code has access to (i.e. it must be able to import it). An explicit import statement is not necessary.
But there is a serious, confusing issue with nested attributes: you will have a local object instead of the remote object!
What's going on: Say you took care of supplying all needed modules and classes to both server and client. Then you want
to access object.sub1.sub2. What you get is the sub2 attribute of a LOCAL sub1 object instance!
This is no problem when you just want to read the value, but it is a huge problem when you want to perform some actions on this attribute, or change the value!
This will be performed in your client, not in the server object!
Explanation: as pointed out above, Python searches for sub1 in object. Because Pyro intercepts the getattr call,
you get the remote sub1 object. Its state is pickled on the server (by Pyro) and unpickled on the client (by Pyro), thereby
recreating the remote sub1 object on the client. Only after that, Python looks up sub2 in sub1.
What happens is that Python itself returns you the sub2 attribute of the local sub1 object!
If you call methods on that object, or change it's value, that will only happen on the (temporary) local sub2! All changes are lost
because they're not propagated to the server!
Currently the only working solution for this problem is to use extra methods on your original Pyro server object, that manipulate the sub2 object on the server (for instance,
you should call object.setSub2(newvalue) to update sub2's value on the server, instead of object.sub1.sub2=newvalue).
Note that this problem only occurs when you're using nested attribute access. Manipulating the attributes of object itself (the Pyro proxy) is no problem,
because all attribute accesses (read and write) are intercepted by Pyro and are remoted to the actual server object.
ConnectionClosedError or even ProtocolError) it can use a special 'hidden' method
of the internal Pyro protocol adapter object to request it to reconnect
to the server:
yourpyroobject.adapter.rebindURI()
tries and
wait. The first is the number of retries that is performed
(default: sys.maxint), the second is the delay time between each retry
in seconds (default: 1).
connectPersistent
method of the PyroDaemon. Otherwise the reconnect feature does not work because the client tries to access an invalid URI.
The connectPersistent method reuses the object's URI that is still known in the Name Server.
It's no problem to always use connectPersistent instead of the regular connect,
but there is a naming lookup overhead per call.
rebindURI method of the object's
protocol adapter itself if this is the case.
About the difference between the exceptions that you can catch:
You get a ConnectionClosedError when Pyro was able to establish a connection
initially, but the connection gets interrupted later (because of network problems
or something like that).
You get a ProtocolError ('connection failed') when Pyro was unable to establish
a connection at all, for instance, when the Pyro daemon on the server isn't
running or reachable at the time Pyro tries to establish a connection.
It is debatable if we should talk about reconnection in the second case,
because there has never been a connection to begin with. But, if you want to
handle this case too, just catch a ProtocolError instead of ConnectionClosedError.
Examine the example code provided in the "autoreconnect" example, and Pyro's internal adapter code, to see how the rebind feature works.
ImportError. However, Pyro intercepts this and returns a NoModuleError.
Unless you enable PYRO_MOBILE_CODE. Now, Pyro internally returns a special code, which makes the client submit the missing Python code to the server. Pyro then proceeds as if nothing was wrong, i.e. it
can now load the previously missing module and call those objects! This happens automatically, you only have to enable mobile code on the server by setting the PYRO_MOBILE_CODE config item to 1!
There is one more thing: loading and running arbitrary code is dangerous. Pyro has codeValidators to help you protect your program; see the Security chapter for more info.
Pyro supports 2-way mobile code: your client can also receive code from the server that wasn't available before!
This means the server can return objects that are unknown in the client. The module(s) on the server that
define those objects will be sent to the client to make them available there, too. Just like enabling mobile
code support on the server, you have to enable PYRO_MOBILE_CODE on the client to allow
mobile code downloading from the server to the client. The codeValidator on the server checks
both directions.
There are a few important limitations with the current mobile code implementation:
__main__ will not work because of a Python peculiarity.
There are no plans for including a workaround in Pyro.
__init__, you have to import them from within the mobile object class methods.
It is perfectly ok to put your mobile object module in a Python package.
If your codeValidator allows it, your mobile object module can also import other
modules that are unknown on the server (but only from within the mobile object class members, not
at the module level or from __init__). They will be transported too (if
you import them by their fully qualified name).
It's easy enough to write an appropriate codevalidator.
If you don't want cascaded loading, check for specific module names.
If you want to allow cascaded loading, check for module name patterns,
for instance allow everything that starts with "agent.".
Have a look at the "agent2" example to see how this all works.
Note: if a compiled module (*.pyc or *.pyo) is available, Pyro will use that. If you have old *.pyc files around made with a different Python version, Pyro will crash (with a syntax error) because it can't recognise these compiled files and tries to compile them! Be sure to delete all *.pyc and *.pyo files if you switch Python versions.
Another case where multithreading is necessary, is when you are using callbacks in your Pyro object (or when you are calling other Pyro objects). Because this call may arrive at the same daemon, it must create a new thread to handle the new incoming call. When running in single threaded mode, Pyro will freeze because the calls are waiting on each other to be processed.
You can use multithreading in your clients, but you cannot share
Pyro proxy objects between threads. Each thread has to create
its own proxy object. Probably the easiest way to do this is to just look up the required object from
inside every thread. You can also look it up once, and use the copy module
to let every thread create a clone of the proxy object: copy.copy(obj).
Alternatively, you can use thread-locking around the remote calls.
Pyro's use of threads on the server is as follows: the main daemon loop only waits
for new connections. If a new connection arrives, a new thread is
created that will serve this connection. The thread will process
incoming calls sequentially, but that is good ofcourse since they can
only arrive sequentially over the socket. The thread keeps running while
the connection is still there. Be aware of this: if you ^C a program,
you abort the main loop but many other threads might still be running.
Pyro makes them 'daemon threads' so they will be terminated when your
program exits, but it is preferred to clean up the nice way:
call daemon.shutdown() when your program exits.
Of course, a little overhead is introduced. You can see this quite clearly
when you are running the "benchmark" example in single- and multithreaded mode.
Pyro will default to the multithreaded mode if your system supports it,
because usually you'll need Pyro to be able to accept new invocations while
others are still in progress. If you want, use the
PYRO_MULTITHREADED config item to switch to singlethreaded
mode (set it to zero). Pyro will default to single threaded mode if Python
threads are not available. Switching to multithreaded mode if Python
threads are not available, by setting
PYRO_MULTITHREADED to 1, will crash Pyro.
Please use Pyro.util.supports_multithreading() to test whether or
not your system is able to use multithreading.
Pyro.core.SynchronizedObjBasePyro.core.SynchronizedObjBase base class that you can use instead
of the regular Pyro.core.ObjBase. When you use it, all (remote) method calls are automatically synchronized
for you, because a thread lock object is used for your Pyro object. This has a huge negative impact on heavily
multithreaded applications, but it saves you from much threading headaches.
Note that other Pyro objects may still be accessed concurrently. If you share data over different Pyro objects,
you still have to make sure that everything behaves in a thread-safe manner.
The "multithread" example shows what's going on, and how multithreading can help to improve performance and response times.
__init__ method for instance).
How do you gain access to the TLS? The ObjBase base class
has a method getLocalStorage() that returns an instance of a storage class to put your data in.
(It also works on platforms that don't have threads).
The TLS object has one predefined attribute: caller. This attribute contains
the Pyro.protocol.TCPConnection object of the client that is performing the current method call.
It is advised to leave this alone, but you could do nasty things with this if you want to (such as
dropping the connection by calling close() on it).
You might also use the caller object to access any specific data associated with the connection,
that you have stored there yourself (for instance, the authentication id placed as
an attribute on it from within a custom connection validator). See the "user_passwd_auth" example.
How do you initialize the TLS?
Pyro allocates the TLS as soon as it's needed. You cannot initialize it
when your objects are initialized because it doesn't exist yet at that time.
Instead, you have to set a custom init function in the Daemon using the
setInitTLS(initfunc) method of the Daemon. initfunc
must be a callable object that takes a single argument: the TLS object to
initialize.
When running in multithreaded mode the TLS is unique for each caller thread ofcourse, but also for each object: every Pyro object has a different TLS. But it's best not to depend on that feature because it might change in the future and it is already not true when you run in singlethreaded mode. In singlethreaded mode the TLS is an object allocated in the Daemon, and it is shared across every Pyro object and every method invocation.
If you use delegation instead of inheriting from ObjBase, you cannot use TLS because you don't have a means of getting to it
(the getter is in ObjBase). This isn't bad because objects
that use delegation know nothing about Pyro in the first place.
(Well, actually, there is a sneaky way of getting to the TLS:
use threading.currentThread().localStorage (it's an attribute
of the current thread object). But if you do
use this, be aware that your delegate object becomes dependent on the Pyro
environment, and you probably chose to use delegation approach to avoid this!)
Also, in a single-threaded environment, this is not possible because
the threading.currentThread() call might not be available. In
any case, when Pyro is running in single threaded mode, the current thread
(or main thread) does not contain a localStorage attribute.
To avoid any problems, it is probably best not to use it at all. Remember
that you can use self.getLocalStorage() fine from a Pyro object
that is inherited from Pyro.core.ObjBase, even in single threaded mode!
Usually Pyro will locate its servers and objects using fixed IP numbers
encoded in the Pyro URIs. This may or may not be appropriate.
For instance, when a machine has an IP number that is only temporary, such
as DHCP-leased IP numbers. The machine can get a new -different- IP number while
Pyro is still running. URIs with the old IP number are now invalid!
Therefore it is possible to tell Pyro to use symbolic DNS hostnames
in URIs instead of raw IP numbers. Pyro will then use DNS to look up the
actual IP number of the specified host (by name). You can enable this by setting
the PYRO_DNS_URI config item to 1. However note
that each time Pyro has to look up a host, there is a DNS lookup delay.
If your machine has multiple IP addresses (for instance, when it has
multiple network cards), you have to decide on what IP address your
Pyro servers reside. When you create a Pyro Daemon, use the host
argument to specify the hostname/ip address to bind the server on (defaults to '' - the default host).
The Name Server can be started with a -n hostname argument, to
specify the hostname/ip address to bind on.
Another issue is when you're using Pyro behind a firewall.
There is one specific form of firewalling that is addressed in Pyro:
simple Network Address Translating firewalls using port forwarding.
Let's say you're at 192.168.0.1 (a private address) behind a NAT
gateway that's 192.1.1.1. You have port forwarding on the NAT, so
that Pyro requests go to the private box. However, with the way that
Pyro is set up by default, the daemon will publish URI's as though they come
from 192.168.0.1 -- an address unavailable to the rest of the Net.
There is no way to have 192.168.0.1 publish URI's as though
it were actually 192.1.1.1. Were it not for the extra publishhost
parameter for the constructor of Pyro.core.Daemon.
When constructing a Pyro Daemon, you can give it a special hostname or IP address that it should use when publishing URIs, via the publishhost parameter. The host parameter still is the "real" hostname
of the machine the daemon is running on. When publishhost is
not specified (it isn't by default) the value for host is taken.
If that isn't specified either (it isn't by default) the hostname of the
machine is queried and that is used. In our little example, host
should be 192.168.0.1 (or just empty/omitted) and publishhost should be 192.1.1.1, the address of the firewall/gateway.
Your client must publish a callback object that is a true Pyro object. In fact,
for the callback part, your client must act as a true Pyro server. So you have to
code your client as both a client and a server.
This may require that your client must run a separate thread to handle Pyro
messages (the Pyro daemon loop must be running to accept incoming calls).
If your client only sits idle, and only waits for incoming callbacks, you can
just run the daemon's requestLoop in the main thread.
Have a look at the "callback" example for more details.
Be very aware of threading issues. Callbacks occur in their own thread. A callback may even occur while your client is still busy processing the previous callback. Your server should be even more prepared for callbacks. Usually you have a method that "registers" a client as a callback object. The client will call this method and has to pass a proxy to the callback object, not the object itself! (see usage rules, below).
Possible deadlock: if your objects enter a conversation, deadlock may occur easily. For instance, A calls B, B calls back to A, A calls B again... deadlock! B was still waiting for A to answer the callback, but A invokes a new method instead. Pyro cannot handle this yet. This issue might be addressed in a future Pyro version. In the meantime, please read on about possible alternatives.
Your server usually has a list of callback objects that it should call. Be very careful here:
a client can unexpectedly disappear. You have to handle ConnectionClosedError
exceptions and you must then remove the defunct callback object from the list.
But you have to do this in a thread-safe way (the list must be under a thread-lock),
because the server may be multithreaded!
The server also has to handle any exceptions that occur in the callback object on the client.
Don't trust it. Catch any exception that occurs, otherwise your server dies.
Be aware of a complex issue when your callback object raises an exception:
if a callback occured in a remote method, that was
called by the client, the exception might travel right back to the client
if the server doesn't take precautions!
Not strictly callbacks, but when your Pyro object is calling other Pyro objects, you have to run in multithreaded mode. Because the new call may arrive at the same daemon, it must create a new thread to handle the new incoming call. When running in single threaded mode, Pyro will freeze because the calls are waiting on each other to be processed.
Please consider using the Pyro Event Service, instead of custom callback objects. It will handle all those nasty things for you, at the cost of some control. But it's very easy to use.
Special callback object Pyro.core.CallbackObjBase: Usually any exception that occurs
in the callback object is silently transported back to the server, where it is raised again.
For many callback objects, this is not exactly what you want, because usually
the client is interested in an error within a callback object, and
the server often doesn't care. If you use the special Pyro.core.CallbackObjBase
as a base class for your callback objects (instead of the regular ObjBase),
any exception that occurs in the callback object is not only sent to the server,
but also raised again on the cient. You can see this work in one of the "callback" examples.
Oneway calls: These are remote method calls that do not expect any answer, so they return immediately after sending the remote call to the remote object. The call does not wait for the remote method to finish. At a lower level, it doesn't even wait for a protocol reply, so performance is much better too. On the server side (the object that executes the oneway call), the call is executed in its own thread so that other calls can be processed in the meantime, if the oneway method takes some time to complete. This only works if multithreading is enabled. This property of oneway calls allows you, for instance, to start a computation and continue your own business while the computation runs. Later on you use another remote call to retrieve the results of the computation (or use a callback). This way, you don't have to worry about programming a multithreaded client, just use a oneway call.
But! There is no way to find out what the result of your request was - whether it succeeded or failed. No result nor any exception is ever returned. You're still quite sure that the call is performed though, because the request is sent using the regular PYRO protocol, but there is no guarantee (nor is there with regular calls by the way).
Oneway calls are very nice to have in a callback scenario. The Event Service also makes heavy use of them. Why? Your server is freed from the burden of handling exceptions that may occur in the remote method, and it doesn't block on slow or buggy clients. It just sends out the method invocations and continues on happily while the callback clients process the incoming method call.
You have to specify at runtime in your program which methods of what objects have this Oneway semantics. You do this by calling a special method on the Pyro proxy object:
obj._setOneway(methods)where obj is your proxy and methods is a list or tuple of method names that have to be called Oneway. It may also be a single method name. Currently there is no way to specify from within the remote object itself, or the creating process, that a method has to be called oneway. The calling party has to set this property. Ofcourse you could build some sort of inquiry method that has to be called first and that tells the caller what methods can have this property, and maybe this will become automatic in a future version, but it's not yet there.
Assume the remote Pyro object raises a ValueError exception.
Your calling code will receive this and crash wit the same ValueError exception.
However, the stacktrace in the traceback info is from your local
code, not from the remote code. If you're just catching and processing exceptions,
and don't want to deal with stacktrace/traceback info, there is no problem with this.
But if you want to print the stacktrace, it is meaningless! It is a stacktrace from
within the bowels of the local Pyro code! It provides no clue what piece
of remote code caused the problem.
To help you with this, Pyro puts the remote stacktrace inside
the exception that travels to your calling code. It can be obtained from
the special attribute that is defined in Pyro.constants.TRACEBACK_ATTRIBUTE,
but it's probably more convenient to use the utility function
Pyro.util.getPyroTraceback. You pass the exception object and
the function returns a list of lines that contain the remote traceback
(if available) and the local traceback. For example;
try: print thing.method() # thing is a Pyro proxy except Exception,x: print ''.join(Pyro.util.getPyroTraceback(x))This function is safe to call on all exceptions, also normal (local) exceptions. See the "simple" and "exceptions" examples.
Actually, to make it even simpler, the exception objects that Pyro raises have a customized __str__
method. If you print them or otherwise convert them to a string, they automatically include any remote
traceback information in the traceback text. So, strictly speaking, it isn't even necessary to
code an exception handler to call getPyroTraceback, you can just treat the exceptions
as any other. If they are raised by Pyro, and contain remote traceback info, you get that
info transparently. Your code doesn't need to know if it's dealing with Pyro exceptions or not.
Pyro.constants.INTERNAL_DAEMON_GUID.
(this is not a GUID in the true sense, because it is not unique). Here's an example
to query all registered objects on a remote Daemon at 192.168.1.50, running on the default port:
import Pyro.core Pyro.core.initClient() d=Pyro.core.PyroURI(host='192.168.1.50',objectID=Pyro.constants.INTERNAL_DAEMON_GUID).getProxy() print d.getRegistered()The result is a dictionary that maps GUIDs to object names, like this:
{'c0a8013208585602469aec911dc92a20': ':Pyro.NameServer', 'c0000000011000001000000010000001': '__PYRO_Internal_Daemon'}.
Currently there is only one other daemon method that is remotely accessible, ResolvePYROLOC.
You will probably never call this yourself, it is used for the PYROLOC: protocol.
Note: If the daemon is running in SSL mode you have to add an additional prtcol="PYROSSL"
argument to the PyroURI constructor call above!
Because Pyro's connection protocol requires a handshake at connection time, the timeout also works for connecting to a Pyro daemon. This is nice, because evil clients now cannot eat connections indefinately by just connecting and never finishing the request. Once a connection has been established, it stays there.
How do you set the timeout?
proxy._setTimeout(20) # set 20-sec. timeout on proxy object (client) daemon.setTimeout(20) # set 20-sec. timeout on daemon (server)Clear it again by passing
None. If a timeout occurs, Pyro raises a TimeoutError exception,
which is subclassed from ConnectionClosedException. The connection has already been dropped by Pyro.
(why is it _setTimeout -- with an underscore -- for the proxy? To avoid possible name clash with your own method names)
Note about NS and ES: the Name Server and Event Server have a built-in fixed timeout of 20 seconds. The connection is killed if data transmission takes longer than that, to prevent evil clients of clogging up the servers.
proxy._release() to release
the connection. Pyro will automatically create the connection again as soon as you start using the proxy again.
An additional benefit is that the socket object associated with the proxy is destroyed, so you are able to pickle the proxy object again.
Things are a bit more complex on the server. Usually the Pyro objects you create there must
stay active because it is not known when a client comes by to use them. This is especially true
for objects that are registered in the Name Server, because that sort of tells the clients
"this object is available there-and-there". But for transients, things are
different. Remember that transients are objects created on the server, without a name,
and usually a proxy is returned to the client to access these new objects. There's no way
to tell when the client no longer needs the object! If it 'forgets' to call some sort of
"release" method that destroys the object in the server, your server could overflow
with all left-over unused transient objects. This is why you can specify an inactivity
timeout for transients, by calling daemon.setTransientsCleanupAge(timeout)
on your daemon. The argument is in seconds.
Pyro tracks the time an object has not been used, and destroys it when the timeout has expired.
You may want to override the _gotReaped() method in the ObjBase to
act on the destroy event (for instance, remove the object from a list).
Please be aware that if Pyro runs in single-threaded mode, the reaping of expired objects is only
done when a method is invoked!
Also, cleaning up objects on the server doesn't automatically release any network
connections and threads that might have been created for these objects. This is because these
resources are not necessarily associated with a unique object, so they have to remain active.
See also the "Bank2" example, it uses timeouts.
__init__ method. You should use a regular initialization method that you must call explicitly after
binding to the remote object. The __init__ method will only be called on the server side when the object is created.
openFile(filename) that opens
a file on the server and returns the file object, because file objects cannot be pickled. This also holds for sockets, and various other object types.
As long as you don't access these objects in your client, you're OK though (there's nothing wrong with a Pyro object that
has various open files or sockets as attributes -- if you don't access them from the client).
object1==object2 for instance. This is because the
proxy class needs to hijack the rich comparison mechanism to be able to compare two proxy classes with each other.
Pyro.naming.NameServerProxy. When you use the Locator, you're safe.
.py source files that contain the code of the objects that are used as parameters in remote method calls, must be available on the client and on the server. Otherwise the server cannot load the implementation code of an object that arrives in a remote method call.
This is no longer necessary if you enable the mobile code feature.
See the "agent2" example.
.py source files that contain the code of the objects that are used as attributes of Pyro objects on the server, must be available on the client also.
Otherwise the client cannot recreate the server object's attributes and will crash if you
access these attributes. This cannot be fixed with enabling mobile code yet.
Pyro.core.ObjBase.
You could define a new (probably empty) class in the server that inherits
both from Pyro.core.ObjBase and the class that is made remotely
available.
Pyro.core.ObjBase. This works as follows: create an object of your remote class, create a Pyro.core.ObjBase object, and tell the latter to use the former as delegate:
... impl = MyRemoteClass() obj = Pyro.core.ObjBase() obj.delegateTo(impl) ...and you then connect
obj to the Daemon.
copy.copy(obj) to create
a clone of the proxy. Alternatively, you can use thread locks around the remote calls.
daemon.shutdown() when your program exits. This will
cleanly stop all threads that might still be running.
__init__ from the base class in the __init__ method -if you have one- of your remote class.
You cannot omit this, your Pyro object will not work without the initialization:
def __init__(self):
Pyro.core.ObjBase.__init__(self)
...
Pyro.core.ObjBase, or use the delegation approach,
and call Pyro.core.ObjBase.__init__(self) from your own __init__
URI = self.getDaemon().connect(object) # no name, not in NS URI = self.getDaemon().connect(object,'name') # registered in NS with 'name'
return object that you would otherwise use, you do:
return object.getProxy() # regular dynamic proxy return object.getAttrProxy() # dynamic proxy with attribute support
self.getDaemon().disconnect(object) del object
while obj:.
Like the limitation with direct member access, this is also caused by the
way the proxy object is currently implemented. It intercepts each and every
method call. And testing for zero-ness or nonzero-ness or coercion are also method calls! (__nonzero__, __coerce__ etc.)
Pyro.core.CallbackObjBase instead of
the regular ObjBase. (see above at "callbacks")
__nonzero__. For example, to make obj+=50 work,
you need to implement the __iadd__ method in your Pyro object and let it return the object itself (that is,
a proxy for the object!) because the method returns the in-place modified object!
def __iadd__(self,value):
self.value+=value
return self.getAttrProxy()
daemon.shutdown() when it exits.
Usually this is not a problem because your program creates a deamon and calls its requestLoop.
But a situation might arise where you don't keep a reference to the daemon object, and then things break.
NameServerProxy) can be pickled, and therefore you can
pass them around within your Pyro system. If they have been used already (and have a socket connection),
the connection is automatically released before pickling. But a proxy obtained from a pickled one can be used right away, because
it will recreate the socket connection on first use. Ofcourse a pickled proxy reconnects to the exact same URI as before,
so it won't work if the Pyro object changed.
daemon.disconnect(object)
| <previous | contents | next> | Pyro Manual |