{"id":2854,"date":"2022-05-25T11:44:10","date_gmt":"2022-05-25T02:44:10","guid":{"rendered":"https:\/\/jennifersoft.com\/en\/?p=2854"},"modified":"2022-05-25T11:44:11","modified_gmt":"2022-05-25T02:44:11","slug":"2022-05-25","status":"publish","type":"post","link":"https:\/\/jennifersoft.com\/en\/blog\/tech\/2022-05-25\/","title":{"rendered":"JENNIFER Python"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

Python language, which has been receiving attention lately in the data processing sector, naturally expended the development experience to the web by the developer, and in addition to this, services using web frameworks such as Django and Flask started to settle down as part of the micro service even in the enterprise environment.<\/p>\n\n\n\n

In accordance with such changes, JenniferSoft released the first version supporting monitoring for the Python environment. This document explains the unique features and operating methods of the JENNIFER Python agent.<\/p>\n\n\n\n

Supporting Environment<\/h2>\n\n\n\n

It is well-known that Python language is platform-independent. But from the perspective of APM products, it is featured by more restrictions compared to the Java\/.NET, etc. environments because of the active development of the Python language specifications and its surrounding run-time environment.<\/p>\n\n\n\n

Supports only Linux<\/h4>\n\n\n\n

Strictly speaking, frameworks such as Django\/Flask can be executed irrelevant to the platform. But most service environments are hosted through exclusive web applications that support WSGI interfaces such as uwsgi or gunicorn, and the problem here is that this application program only supports Linux.<\/p>\n\n\n\n

The fundamental reason for this is because of Python interpreter\u2019s full lock known as \u201cGIL (Global Interpreter Lock.\u201d A single thread that acquired GIL can process the byte code of Python. This is not a big problem in normal environments, but in the case of web application environments that process multiple requests based on threads, severe performance drop issues occur due to GIL.<\/p>\n\n\n\n

In order to bypass this, the WSGI web application (uwsgi, gunicorn, etc.) selected the method of dividing the process with a fork instead of using threads for Python. Therefore, Django\/Flask can be executed in the Windows environment as well, but uwsgi\/gunicorn that hosts it cannot support Windows (due to the use of fork), and thus, JENNIFER Python currently only supports monitoring of the Linux environment.<\/p>\n\n\n\n

Supporting specific Python run time and packages<\/h4>\n\n\n\n

Python language does not have good sub-compatibility for version upgrades. The JENNIFER Python module uploaded within the process also uses Python language internally, and therefore, it is affected by sub-compatibility because it has the same run-time restrictions.<\/p>\n\n\n\n

This problem is applied not only in the Python run-time but also for the package as well. As most packages do not have sub-compatibility and because the features of APM products require close interworking with the internal codes of the target package, it has the problem of having to add support for each package version.<\/p>\n\n\n\n

In result, the Python web application operated by the client website must meet the run-time and package version conditions supported by JENNIFER Python products to monitor at desired levels. If not, it requires additional time for adding functions to JENNIFER Python through prior negotiations for packages not yet supported or its ensuing version.<\/p>\n\n\n\n

Operating Structure<\/h2>\n\n\n\n

As explained above, uwsgi or gunicorn, etc. that host the Python web application program has operating methods based on forks by default. It therefore has to adopt a structure different from existing Java\/.NET agents that are activated in single processes.<\/p>\n\n\n\n

In other words, the JENNIFER module must be loaded for operation for each forked process, but each process cannot be identified as a single instance in JENNIFER View. Thus, it requires a separate process that represents the single application program, while communicating with the fork process. JENNIFER Python reflects this to be activated through the following structure.<\/p>\n\n\n\n

\"\"
[ Figure 1. JENNIFER Python Operating Structure for 1 Web Application Program ]<\/figcaption><\/figure>\n\n\n\n

The proxy process communicates through the JENNIFER data server and single TCP connection, and it communicates with the internal fork process via UNIX Socket.<\/p>\n\n\n\n

Thanks to this structure, the monitoring screen shown in the JENNIFER Console is not too different from Java or .NET.<\/p>\n\n\n\n

Installation and Uninstallation<\/h2>\n\n\n\n

The Python ecosystem has a well-organized package archive called PyPI. JENNIFER Python is also distributed here and currently, it can be read in the following address.<\/p>\n\n\n\n

jennifer-python: https:\/\/pypi.org\/project\/jennifer-python\/<\/pre>\n\n\n\n
 Therefore, the installation uses pip just like other Python packages.<\/p>\n\n\n\n
$ pip install jennifer-python<\/pre>\n\n\n\n
 The first thing to do after installation is to create an INI configuration file containing connection information to the \u201cJENNIFER data server.\u201d This file can be written manually as well, but the basic INI file can be created easily with the following using the \u201cjennifer-admin\u201d command installed together with the \u201cjennifer-python\u201d package.<\/p>\n\n\n\n
$ jennifer-admin generate-config<\/pre>\n\n\n\n
 Then the \u201cjennifer.ini\u201d file is generated with the following contents based on the directory from which the command was executed.<\/p>\n\n\n\n
[JENNIFER]\n\nserver_address = 127.0.0.1\nserver_port = 5000\ndomain_id = 1000\ninst_id = -1\nlog_path = \/tmp\/jennifer-python-agent.log\n<\/pre>\n\n\n\n
 Afterward, the values for each field can be set according to the following criteria using the editor.<\/p>\n\n\n\n
\n\n\n\n\n
 Field<\/td>\nDescription<\/td><\/tr>\n
server_address<\/td>\n JENNIFER Data Server IP<\/td>\n<\/tr>\n
server_port<\/td>\nJENNIFER Data Server Port (default 5000)<\/td>\n<\/tr>\n
domain_id<\/td>\nDomain ID (ID composed in Data Server, default 1000)<\/td>\n<\/tr>\n
inst_id<\/td>\nIdentifier of the monitored Python application program (number within the int16 range) Set as -1 and JENNIFER Data Server decides automatically<\/td>\n<\/tr>\n
 log_path<\/td>\nPath of log life<\/td>\n<\/tr>\n<\/tbody><\/table><\/div>\n\n\n\n
 The jennifer.ini file must be created for every Python application file, and just the inst_id value can be set differently.<\/p>\n\n\n\n
 The Python application program subject to monitoring through the corresponding INI file configuration must be executed by connecting to the jennifer-python module. At this time, by using the method of setting the INI file path together with environmental variables, the Python web application program can be executed in the following format.<\/p>\n\n\n\n
$ JENNIFER_CONFIG_FILE=<Configuration file path> jennifer-admin run <Existing python execution code><\/pre>\n\n\n\n
 If your application program was executed with the existing \u201cuwsgi -i uwsgi.ini,\u201d and the created jennifer.ini file path is \/home\/user\/jennifer.ini, then it can be executed as follows.<\/p>\n\n\n\n
$ JENNIFER_CONFIG_FILE=\/home\/user\/jennifer.ini jennifer-admin run uwsgi -i uwsgi.ini<\/pre>\n\n\n\n
 If all settings are correct, the following log message will appear on the uwsgi execution screen.<\/p>\n\n\n\n
\u2026[Omitted]\u2026\n---------------- [App Initialized] ----------------------\nMachineName =  TESTPC\nIs64BitProcess =  ('64bit', 'ELF')\nPython Version =  3.8.10\nJennifer Python Agent Install Path =  \/usr\/local\/lib\/python3.8\/site-packages\/jennifer\nJennifer Python Agent Domain ID =  2790\nJennifer Python Agent Inst ID =  601\nJennifer Python Agent Pid =  4925\n---------------------------------------------------------\n\u2026[Omitted]\u2026\n<\/pre>\n\n\n\n
 Meanwhile, to download the agent from the web application program, simply execute using the existing method without going through \u201cjennifer-admin.\u201d In addition, to delete the JENNIFER Python binary, simply remove it using pip.<\/p>\n\n\n\n
$ pip uninstall jennifer-python<\/pre>\n\n\n\n
Support Function<\/h2>\n\n\n\n
 Monitoring shown in the JENNIFER Console screen satisfies Active Service view and X-View profile, thus allowing monitoring a similar experience as existing JENNIFER Java\/.NET at basic levels.<\/p>\n\n\n\n
\"\"
[ Figure 2: 3 Instances that installed Python Agents]<\/figcaption><\/figure>\n\n\n\n
\"\"
[ Figure 3 : Profile Verification through See X-View details ]<\/figcaption><\/figure>\n\n\n\n
 The profile subject currently supports HTTP and DB summoning as follows at the time for which this was written (5\/20\/2022).<\/p>\n\n\n\n
  • urllib, requests<\/li>
  • mysql, redis, mongo db, sqlite3<\/li><\/ul>\n\n\n\n
     This is still in its early version and many functions are not yet available in the existing JENNIFER Java\/.NET product, so please be patient as we make many improvements continuously.<\/p>\n","protected":false},"excerpt":{"rendered":"Python language, which has been receiving attention lately in the data processing sector, naturally expended the development experience to the…","protected":false},"author":20,"featured_media":2869,"comment_status":"closed","ping_status":"closed","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[78],"tags":[],"class_list":["post-2854","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech"],"acf":[],"_links":{"self":[{"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/posts\/2854","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/users\/20"}],"replies":[{"embeddable":true,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/comments?post=2854"}],"version-history":[{"count":9,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/posts\/2854\/revisions"}],"predecessor-version":[{"id":2872,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/posts\/2854\/revisions\/2872"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/media\/2869"}],"wp:attachment":[{"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/media?parent=2854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/categories?post=2854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jennifersoft.com\/en\/wp-json\/wp\/v2\/tags?post=2854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}