PEP-578 Audit Hooks for Stackless

[akruis]

C-Python 3.8 adds various Audit Hooks, see. Various Stackless APIs could be used to bypaas those hooks. Therefore Stckless sould add appropriate calls of PySys_Audit() to relevant C-functions (mostly __reduce__() and __setstate__(), tasklet.frame, tasklet.cstate, cstack.\_\_str\_\_, cstack.startaddr, cframe.*, ).

C-Python Hooks to be considered

• code.__new__
• sys._current_frames
• sys._getframe
• sys.set_asyncgen_hooks_finalizer
• sys.set_asyncgen_hooks_firstiter

Stackless already creates the events sys.setprofile and sys.settrace

[akruis]

I created pull request #304.

I fixed the documentation for auditing events sys.setprofile and sys.settrace added in issue #300 to use the Python spacific Sphinx syntax .. audit-event::.

From https://ep2019.europython.eu/media/conference/slides/8MGqsQG-auditing-hooks-and-security-transparency-for-cpython.pdf:

Tips:
• Prefer C call (PySys_Audit) when possible
• Prefix with your module (import) name
• Audit after validation, before execution

I tried to follow these recommendations.

_stackless._wrap.code.__new__() calls CodeType.__new__() and therefore already raises auditing event code.__new__.

I added code to raises the following auditing events:

• Stackless APIs returning alive frames now raise auditing event sys._getframe.
• If a frame gets unpickled that could be evaluated, Stackless raises stackless.frame.__setstate__.
• If an unpickled frame has a trace function that could be executed, Stackless raises sys.settrace.
• If an async_generator finalizer gets unpickled by value, Stackless raises stackless.async_generator.set_finalizer.

[kristjanvalur]

I skimmed over the PIP but still am not entirely sure what its for. Is it security? Or just monitoring?
The events you have added to Stackless, were they selected for a particular reason? Are they significant in any particular way, for example, security issues, or did you just add them to make use of the system and say "stackless here, doing its thing?"
Maybe if you could briefly explain how this pip applies to stackless.

[akruis]

Was a bit distracted from Stackless during the last few days.

I skimmed over the PIP but still am not entirely sure what its for. Is it security? Or just monitoring?

Well both. For security reasons if you operate a large data center (or a could) you want to be able analyze your log files for any anomalies. If an attacker gains control over a Python interpreter, he can do everything and - without auditing hooks - without any traces in the logs. Therefore it is a real advantage, if you can

• retrace basic actions of the program (file and network access, system calls, ...)
• discover unusual / advanced behavior (self modifying code, loading code from unusual or untrusted locations, ...)

PEP-578 is fairly generic and does not specify concrete criteria where to add audit hooks to the code. Therefore I had to find my own criteria:

1. If a C-Python low-level function creates an auditing event and if Stackless provides an alternative to said function, then Stackless shall create the same auditing event too.

   For instance, C-Python adds hooks to all low-level functions returning alive frame-objects (sys._getframe() and sys._current_frames()). Stackless provides additional functions and properties that expose alive frame objects: various __reduce__() functions and tasklet.frame. I added calls to PySys_Audit("sys._getframe", NULL) as if the Stackless functions would call sys._getframe() internally. Any audit framework for C-Python probably knows about the auditing event "sys._getframe". If I would call the event "stackless.tasklet.frame.read" most frameworks would not know how to interpret this event.

2. If C-Python creates an auditing event on a particular operation, Stackless shall create auditing events on similar or analogous operations.

   For instance Stackless can unpickle alive frame objects as part of a tasklet or a generator-, asyncgen- or coroutine-object. Because other ways to load program code create auditing events in C-Python, I added the auditing event stackless.frame.__setstate__. I'm not sure, if I choose a good name for the event.

Up to now I didn't add any auditing events to monitor tasklet scheduling or channel operations. Up to now my applications don't need them and I'm not sure about the criteria when to create them.

Is this reasonable? Should we add this reasoning to the documentation? Where?

[kristjanvalur]

Ok, this explains it well. I think it is reasonable to produce similar auditing events as regular python does for similar activity.
Perhaps we just need a section about auditing, mentioning mostly that which you described above. If we have unique event we should then document it. I think the name you picked is fine, it is in line with the others in c-python.