Embedded python scripting

In a business software with an object model, it is very useful to have an integrated scripting language available. If the integration allows access to the business objects and certain internal Api, a lot of customer-specific automation can be done.

For the first steps with scripting at Vertec, we had relied on Visual Basic Script (VBScript). This was also quite easy, as our applications already provided a COM extension and Vbscript could use it directly. But I was never really enthusiastic about the linguistic possibilities that Vbscript offers. If we wanted to make progress, we needed a “real” programming language.

Luckily Python

At that time, many years ago, Python began to rise as a new star in the language sky. The language was not very well known, but it was rumored that it was being used internally by Google, and that meant something. What appealed to Python at the time was its clear structure, good readability, and module concept. It also had an internal API that allowed embedding. Fortunately, we had the right horse, because Python is now one of the best known and most widely used programming languages.

Embedding of Python

How do we bring Python as a language into our own application? The Python “Extending and Embedding API” is based on the programming language C, which is actually a taboo for us. Unfortunately, there are no ready-made integrations of Python in .NET, apart from own .NET Python variants like IronPython, which unfortunately are no longer really maintained.

This is where our Delphi past comes to our aid. Delphi is a fairly modern high-level language, a variant of the language Pascal, but it can behave towards other components such as C if needed (technically speaking, it supports various calling conventions). This is ideal for a Python integration and in fact there is the wonderful Python4Delphi project written in Delphi, which prepares the project internal Python API for Delphi.  

The Vertec software is mostly based on .NET and written in C#, but for historical reasons some components are still implemented in Delphi and can be called from .NET code. Thus the Python integration is implemented in Delphi and can be used from C#.

The Python4Delphi (P4D for short) library allows you to implement Python modules and classes in Delphi and make them available to Python.

In addition, P4D provides support for hosting embedded Python engines. Normally, a Python program is executed by starting the Python interpreter python.exe and specifying a .py script. In embedded mode, the host program, Vertec in our case, loads the Python interpreter as a DLL library and calls certain API functions on it to start a script. Instead of python.exe, the Python process is called Vertec.Session.exe and does a lot of other things besides executing Python scripts.

Modules and Adapters

Our Python environment includes a built-in Python module as an entry point for scripting code “vtcapp“ which is implemented in Delphi. It provides access to various features to interact with the Vertec software. An example would be the “msgbox“ Function, which with “vtcapp.msgbox(‘Hallo’)“ shows a message box with the specified text.

However, the scripting integration becomes fully usable only with a seamless access to the business objects. For this purpose, we have introduced a generic Python adapter class, which is connected to a Vertec business object. If necessary, such an adapter is instantiated per business object and represents the business object in Python.

Quite Python-like, we have the Python standard methods on the adapters “__getattr__“ and “__setattr__“ of course, accessible as Delphi methods, overwritten and thus allow access to all properties of the Vertec business objects. The Python methods dynamically look up the specified member in the data model and then allow the reading or setting of the corresponding property. Also a “evalocl“ Method for evaluating OCL expressions must not be missing on the adapters.

Python is modular

a major advantage of Python is its module concept. More complex automation projects can be divided into different code modules, which are imported from a central module. Python modules are usually just Python files on the filesystem. This is a bit too open and insecure for our purposes. That is why we extend the Python import mechanism and define our own importers. This is basically done in Python, with certain entry points implemented in Delphi. How this works in general is described in the “Python Enhancement Proposal” PEP-302. We enable the import of “built-in” Python code from the application resources as well as the import of Python modulesmodules which are stored as Vertec data objects.

It also allows us to control which modules are allowed to be imported and which are not. For application security reasons, we do not want to give too much freedom to the code in our scripting environment.

Scope thanks to Context

Since we now use Python in many locations in our applications, it is important that internal access to the Python environment is as simple and transparent as possible. Per Vertec session there is a Python environment that is available throughout the session run. This allows applications where Python objects continue to exist throughout a call. This is very useful for things like caching or the like.

However, we still want to isolate the different locations where Python is used as well as possible. In particular, we want to avoid conflicts due to identical symbols. The Python API provides that when Python code is executed, a dictionary of local variables is passed along. We take advantage of this and introduced the concept of the PythonContext. It is a lightweight construct with its own rate of local variables and access functions, which apply over the reading and setting of Python values in the Python environment and the conversion to the other world (C# or Delphi).

From .NET we can create a Python context in C# and simply run Python statements and expressions on it. Thanks to local variables, the context has its own naming scope in Python. It does not see the local variables of other contexts.

Gift for Power Users

We are constantly expanding the possibilities of our Python integration based on the concepts described. For example, the Vertec reporting system relies on Python to process the data. Of course, we also create a certain complexity that needs to be digested. Customizing Vertec with Python is certainly a tool that is primarily aimed at power users.

In any case, the possibilities for adapting a Vertec environment for the customer are (almost) unlimited.