Pythonエージェントのリリースノート

言語バージョン3.7のサポートを終了しました。

mysql-connectorを最新バージョンに更新しました。

言語バージョン3.12をサポートするようになりました。

有効な設定および設定のソースを報告するようにしました。

共通設定にagent.logger.stdoutのサポートを追加しました。

sys.monitoringのINSTRUCTIONレベルのイベントを有効にしました。

エージェントログのパスがSTDOUTに設定されている場合に、有効な設定をディスクに記録しようとすると例外がスローされていた問題を解決しました。

ハードコードされたキーリストからSCHEMAを外すよう更新しました。

手動によるミドルウェアの設定を使用してDjangoアプリケーションにエージェントを組み込んだ場合、ルートが自動検出されていませんでした。(PYT-3085)

DjangoアプリケーションでのAssess実行時に、 エージェントがURLスキームを信頼性のないデータのソースとして誤ってマークしていました。 (PYT-3082)

FastAPIでの正常な静的リソースへのアクセス時に、エージェントがパストラバーサルを報告していました。(PYT-3021)

Assessモードでリクエストの解析時に、エージェントでメモリリークが発生していました。(PYT-3050)

スレッドIDを取得するのに最新規格を使用するようログ機能を更新し、JSON以外のレスポンスでContrastサーバからのログメッセージを改善しました。

Contrastランナーを使用してGenshiを使用する際に、テンプレートに%があるとエージェントで例外が発生していました。(PYT-3005)

既知の安全なパーサでlxml.etree.fromstringが呼び出された場合に、AssessでXML外部実体参照(XXE)の脆弱性が誤って報告されていました。(PYT-3013)

非推奨の攻撃タイミング機能をContrast Protectから削除しました。

ApplicationProxyクラスをfunction_wrappersと入れ替えました。

リクエストデータのURLの正規化を仕様に基づいて更新しました。

Contrastランナーの詳細なヘルプメッセージを作成しました。

pathlibに伝播ポリシーを追加しました。

接続開始プロセスの完了後に有効な設定をログに記録するようにしました。

失敗したランタイム接続の結果のログを生成するようにしました。

古いバージョンのFastAPIフレームワークテストに対してanyioのバージョンを固定するよう修正しました。(PYT-3001)

CacheControlsRuleの例外があっても、キャッシュ防止制御の脆弱性が報告されていた問題を修正しました。(PYT-2997)

ユニットテストで「Directory not empty」の警告が出ていた問題を修正しました。(PYT-2996)

Contrastランナーをリリースしました。Pythonアプリケーションを検査するための新しいコマンドラインインターフェイスです。

deprecated_middlewareモジュールと関連する全てのコードを削除しました。

ランナーにリライタを登録した直後から、ポリシーベースの変換が適用されるようになりました。

funchookが有効になっていない環境で、文字列フックが早い段階で適用されるようになりました。

ruamel-yamlの依存関係を最新バージョンに更新しました。

requestsライブラリにssrfトリガーポリシーを追加しました。

特定のミドルウェアのクラスが使用されている場合の非推奨の警告を追加しました。

最新のFastAPI(0.96.*)のサポートを追加しました。

ミドルウェアに自動でエージェント組み込みを行うことを無効にする、agent.python.enable_automatic_middleware configurationオプションを追加しました。

M1/ARMアーキテクチャで、エラーを発することなくインストールできるようになりました。

posixpathのポリシーベースの変換(隠し設定オプションを含む)を削除し、伝播ポリシーを追加しました。

typing-extensionsの依存関係を最新バージョンに更新しました。

ミドルウェアAIOHTTP、Bottle、Falcon(ASGI)、Falcon (WSGI)、FastAPI、Flask、Pyramid、Quartにエージェントを自動で組み込めるようになりました。

JSONポリシーのcontrast.agent.policy.loader および未使用のコードを削除しました。

TRACEのログ取得を無くしました。

builtinsモジュールの伝播ポリシーを更新しました。

genericpathモジュールに伝播ポリシーを追加しました。

_codecsモジュールの伝播ポリシーを追加/更新しました。

contrast.__init__を更新して、pkg_resourcesを直接使えるようにしました。

Django用ミドルウェア(ASGIとWSGI)、Falcon用ミドルウェア(ASGIとWSGI)、Pyramid用ミドルウェアを削除しました。

Contrast Protectのレスポンスを上書きしていた機能を削除しました。

Contrast Protectで、HTTPフォームデータからの攻撃が報告される際に、ソースが誤ってクエリ文字列としてリストされる問題を解決しました。(PYT-2974)

vulnpyのパイプラインを修正しました。(PYT-2955)

app.run()に対してQuartのエージェントの自動組み込みを修正しました。(PYT-2929)

app.run()を使用するサーバで起動されるFlaskアプリケーションに、エージェントが自動で組み込まれるよう修正しました。(PYT-2918)

「AttributeError: BoundLogger object has no attribute trace」を修正しました。(PYT-2790)

OpenAI APIのプロンプトインジェクションをトリガーするポリシーを実装しました。

証明書エラーを無視するためのapi.certificate.ignore_cert_errors設定オプションを追加しました。

Python 3.7のサポートに対して非推奨の警告を追加しました。

バージョン管理のためのsetuptools-scmの利用を再検討しました。

_evaluate_body_jsonに文字列以外が渡されるとエージェントがクラッシュする。 (PYT-2930)

テレメトリが有効で接続に失敗した場合、エージェントがメインアプリケーションスレッドをブロックしてしまう。(PYT-2872)

一部の特殊なケースにおけるエラーを回避するためにエージェントの解析機能を更新しました。

ContrastのURLの末尾にスラッシュが設定されている場合、取り除かれるようになりました。

When using the rewriter with an instrumented application that uses Pandas, an exception occurs. (PYT-2856)

When the ASGI scope contained a client value of None, the agent raised an exception during source tracking. (PYT-2853)

Updated PyPI deploy action.

Moved agent initialization out of runner and back into middleware.

Add integration test for Assess dataflow using runner with uwsgi.

Bump typing-extensions dependency to the latest version.

Bump ruamel-yaml dependency to the latest version.

Bump isort dependency to the latest version.

Bump structlog dependency to latest version.

Moved all packages in contrast.extern to new contrast_vendor package.

Removed flask-specific middleware.

Automatic middleware hooks now set and log the detected framework.

Removed fastAPI-specific middleware.

Added policy for urllib.parse propagators and sanitizers.

When the ASGI scope contained a client value of None, the agent raised an exception. (PYT-2850)

When running in Protect mode, the agent incorrectly identified normal header values as attacks. (PYT-2834)

When processing an HTTP 1.1 chunked response, the agent prevents the entire response from being sent to the client. (PYT-2822)

When analyzing certain objects with custom __bool__ implementations, the agent raised an exception. (PYT-2757)

When performing Assess analysis, the agent occasionally failed to correctly determine if an unknown object was iterable. (PYT-2756)

When processing certain JSON request bodies, Contrast Protect did not correctly adjust input types before performing analysis. (PYT-2824)

When analyzing mysql-connector installations missing a C extension, Contrast Assess failed to instrument the entire library. (PYT-2820)

[en] Fixed issue when using the agent in a Python 3.7 environment that does not have typing-extensions installed, the agent logger raises an exception. (PYT-2814)

Removed all bottle-specific logic from BottleMiddleware.

Updated RuntimeError in setup.py to include a workaround for aarch/arm systems.

Removed quart-specific middleware.

Added a lock for agent state initialization.

Added support for the latest FastAPI (0.93.*).

Changed the AgentLib log level.

Integrated the Protect Library.

Fixed AttributeError: Constant object had no elts attribute. (PYT-2740)

Fixed UnicodeDecodeError: utf-8 codec could not decode byte 0xff in position 0: invalid start byte. (PYT-2725)

Fixed TypeError: module, class, method, function, traceback, frame, or code object was expected, got partial. (PYT-2664)

Fixed SystemError: <built-in function in_scope> returned a result with an error set. (PYT-2659)

Fixed flaky pipeline step for server activity. No more flaky failures in flask-agent-direct-teamserver-response. (PYT-2651)

When trying to identify the current platform on startup, the agent raised an exception. (PYT-2777)

When instrumenting Bottle and Falcon applications, middlewares closer to the application than ContrastMiddleware were bypassed. (PYT-2727)

Upgraded externed wrapt from 1.14.1 to 1.15.0.

Upgraded agent-lib to 0.5.2

Added propagation policy for posixpath (os.path)

Bump pinned version of contrast-agent-lib to v0.4.0

Updated copywrite message

Removed the bypass option for messages

Updated flask-sqlalchemy to 3.0.*

Added support for Quart 0.18 / Werkzeug 2.2

When app server exits, Telemetry will send any messages in queue

Deprecated SR

Added support for Python 3.11

AttributeError: 'Settings' object has no attribute 'last_update_service'. (PYT-2563)

AttributeError: 'NoneType' object has no attribute 'disabled_assess_rules'. (PYT-2559)

AttributeError: 'NoneType' object has no attribute 'reportable_format'. (PYT-2552)

When the regex split result contains None, an exception occurs in the propagator. (PYT-2644)

When unicode-escape is used as encoding, LookupError occurs in path-traversal sink. (PYT-2553)

Bump contrast-agent-lib version to 0.5.2.

When too many assess regex propagation events occurred in a single request, the agent continued to propagate regardless of assess.max_propagation_events. (PYT-2676)

When evaluating the protect deserialization rule for pickled data, the agent easily found false positives. (PYT-2646)

When performing Django's safe_join on a PosixPath object, the agent raised a TypeError. (PYT-2592)

When the Contrast Service was too slow to start on server startup, the Agent failed to instrument the application. (PYT-2582)

When the agent failed to connect to the Contrast Service, it raised an AttributeError when handling the first request to the application. (PYT-2574)

When untrusted data was safely passed to subprocess functions using shell=False, the agent incorrectly reported a command injection vulnerability. (PYT-2557)

Added support for URL exclusions when communicating directly with Contrast.

Added configuration support for input exclusions directly from Contrast.

Integrated AgentLib Request End for contrast_c::check_cmd_injection_query

Upgraded externed typing extensions to latest version.

When ASGI websocket endpoints are used with the agent enabled, the app crashes. (PYT-2517)

When starting the Contrast Service without a path, the Agent defaults to STDOUT rather than the default file path. (PYT-2505)

Reduced frequency of server activity messages. (PYT-2484)

When attempting to rewrite the target of an append, the agent incorrectly passes self rather than the original Attribute. (PYT-2496)

Added support for FastAPI 0.85. (PYT-2456)

Implemented PII masking in the agent. (PYT-2462)

Added a CONTRAST__AGENT__PYTHON__REWRITE option that disables rewriting. (PYT-2485)

When running the agent, getting specifications in rewriter throws AttributeError. (PYT-2480)

When running the agent, rewriter cannot find specifications path. (PYT-2476)

Fixed issue for when an agent receives a websocket request and a KeyError was returned. (PYT-2470)

Fixed issue where an application instrumented with the agent in an environment without lxml installed crashed. (PYT-2464)

Falcon ASGI support is now available in Python agent 5.14.0.

Added support for Django 4.

When attempting to rewrite code for instrumentation, certain code patterns result in infinite loops, resulting in application crashes. (PYT-2444)

When agent.service.logger was configured to log to an unwritable path, the Contrast Service wrote DEBUG logs to stdout regardless of the configured log level.

When sampling settings are translated from Contrast, a String vs Enum mismatch causes them to be improperly applied. (PYT-2286)

When sqli protect rule isn't enabled, a NoneType error is logged (Telemetry-reported Error). (PYT-2311)

Telemetry exceptions - production issues from live customer environments will be reported to our servers to help remediate bugs.

Expanded FastAPI support to version 0.75.*.

When assess configuration specifies sampling.baseline, the agent incorrectly analyzes fewer baseline requests than expected. (PYT-2238)

When an assess-instrumented function throws any exception, the agent logs it at ERROR level even when recoverable. (PYT-2234)

Modified the agent to better support asynchronous functionality added by event.

Introduction of the Assess Sampling feature to facilitate performance tuning.

When a Django request's path is missing a required "/", the agent does not mark the route as observed. (PYT-2222)

When Django creates a redirect response to append a trailing "/" to the URL path, the agent incorrectly finds an unvalidated redirect (PYT-2218)

When the agent encounters an issue during assess dataflow tracking, an error level, rather than debug, log message is created, potentially filling log tools. (PYT-2196)

When processing a request without a context, a missing None check results in an AttributeError. (PYT-2168)

When Protect is disabled either locally or in Contrast, the agent disables it. (PYT-2186)

When a Django app serves a jinja template, the agent consumes the response (PYT-2166)

When an aiohttp application in Protect mode receives a ContrastServiceException, an error that crashed the app occurs (PYT-2167)

When an aiohttp, Fastapi, or Bottle app is onboarded, an incorrect framework name is reported (PYT-2171)

The Python agent now supports FastAPI 0.74.

Added web application framework support for AioHttp.

Support for asynchronous web application framework - AioHTTP.

When a protect-instrumented method receives non-UTF-8 encoded bytes, the agent fails to perform protect analysis. (PYT-2028)

When an application calls an instrumented method with a keyword-argument named "self", the agent throws an exception. (PYT-1998)

When handling many concurrent requests in Protect mode, the agent failed to perform path-traversal postfilter analysis. (PYT-1964)

Expanded web application framework support for Pyramid 2.

Telemetry is now enabled in the Python agent in order to gather valuable data about the agent’s functionality. The data is all anonymous, no personal information is collected.

We no longer report vulnerabilities when the value of url_scheme is used in a trigger.

The Python agent now supports the web framework FastAPI.

Expanded web server support for uWSGI.

Expanded web server support for gUnicorn.

Expanded web server support for Uvicorn.

Architectural improvements for propagation and associated extension hooks.

When the agent was installed in Python 2.7, the latest version of the protobuf dependency caused agent startup failure as it dropped support for older Python versions. (PYT-1757)

When a Bottle or Django app received a request with certain encoded chars in the path, the agent raised a UnicodeEncodeError. (PYT-1742)

Implemented event limit specifications to improve performance.

When the agent tried to detect the application framework, it always picked the default one. (PYT-1707)

Performance improvements from refactoring and other optimizations.

When a framework test was executed in an environment in which the agent was installed, then load ordering between the test code and agent code causes an inability to track some objects. (PYT-1574)

Remove specific support of the Pylons framework given its merger with the Pyramid framework.

Expand support of the Flask framework to include the new 2.X major version.

When a vulnerability payload is too large, TeamServer cannot process the finding and rejects it. (PYT-1488)

Added support for built-in sanitization and validation in the Falcon 2.X and Pyramid Web Application Frameworks to improve vulnerability detection when Assess is enabled.

The agent now reports all dependencies used by the application, even those without Python files.

The agent now supports TRACE level logging to assist with diagnostic investigations.

The agent has been refactored to provide performance improvement during vulnerability detection and reporting when Assess is enabled.

The agent now supports MarkupSafe 2.0 as used for cross-site scripting (XSS) vulnerability protection when observing data flow when Assess is enabled.

When customer uploads large file to instrumented falcon route, agent performs too much analysis, causing request to fail to complete. (PYT-1476)

The Python agent now supports the web framework Bottle.

Incremental changes were made to reduce memory usage when Assess is enabled.

Added Assess support through pathlib.Path objects.

Pycassa library is no longer supported by the agent.

Added additional Assess support for Command Injection vulnerabilities in os and subprocess modules.

Added additional Assess support for marshal and shelve modules.

Added additional Assess support for Path Traversal vulnerabilities in os, pathlib, and shutil modules.

Added os.path.basename as sanitizer for Path Traversal.

When propagating through string methods with the KEEP action, Assess fails internally on length - 1 return values. (PYT-1466)

When running Assess on some alpine docker containers, the agent fails to instrument string methods, preventing application startup. (PYT-1450)

The agent now propagates through str multiplication.

The PyramidMiddleware is now a WSGI-based middleware.

User input that is later encoded as Base64 will now have proper dataflow context.

When attempting to instrument strings in Assess, not enough memory is allocated in some environments. (PYT-1350)

Refactored the patch manager for better instrumentation.

The agent will now fully shutdown if told to do so by the service.

Improved agent startup time and memory usage when Assess is enabled.

The agent now reports its effective instrumentation mode to the service.

Upgraded bundled service in the agent to the latest version.

When casting an object with a __str__ method to a non-str, the agent would lose propagation through the cast propagator. (PYT-1312)

When trying to instrument an application using SQLAlchemy, an error would be thrown if MySQLdb had no attribute called Cursor. (PYT-1332)

When trying to retrieve properties for an object, exceptions could be thrown if the agent attempted retrieval in an unsafe manner. (PYT-1333)

Refactored the agent's approach for handling objects with attached properties.

Updated Django support to prevent over tagging of path values within a request.

Upgraded service version bundled within the agent.

Updated dataflow action to prevent potentially unsafe propagation of tags.

When reporting a vulnerability while using legacy Django middleware, the agent would not run its final tasks if a trigger raises an error. (PYT-1257)

When trying to track dataflow through an action on a bytes object, the agent could fail. (PYT-1281)

When trying to format to lowercase hex, the format propagator would fail. We fixed this edge case. (PYT-1227)

Support for Python 2.7 will be deprecated in 2021.

Added support for the Assess ReDoS rule.

Reduced the calls to retrieving Python loaded modules to improve performance.

Improved representation of dataflow with events containing keyword arguments.

The agent failed to process GET requests when handing user input from requests containing non-UTF-8 strings. Updated string handling to account for this. (PYT-1242)

When users uploaded files in a Flask application, the amount of resultant dataflow events in internal web framework code caused significant performance degradation. Given the lack of risk fo exploitation with this method, we expanded ignored methods for Flask, preventing these events and subsequent degradation. (PYT-1247)

Added support for Python 3.9.

Agent does not unpack method arguments when passing them into Assess rules.

Agent now ignores .so files from loaded modules when patching.

Added route coverage support for DjangoRestFramework routers.

Added capability of deadzoning methods to improve accuracy.

Assess stacktraces can now be configured with assess.stacktraces.

Corrected possible string tracker age off KeyError on key deletion.

Service and agent were using different environment variable to set the config.path. Updated CONTRAST_CONFIG_PATH value parsing to look for a file and not a directory with the YAML file. (SUP-2257, PYT-1161)

Added additional debug logging on request start and finish.

Improved library analysis support to include parsing SOURCES.txt.

Updated timing of logging of the environment of an application.

Added logging of configuration values on any logger change.

Fixed propagation scope leak for generators.

Agent crashed in a scenario where free() could be called twice because of patching by Gevent.. (PYT-1164)

The agent now reports enhanced library usage on startup and at the end of each request. After a request is received by the agent, we report new files loaded for an installed package.

The agent will not report unsupported distribution types for packages.

Updated the heartbeat thread to no longer accidentally propagate and cause an error.

Added configurable Django Rest Framework (DRF) response-rendering deadzone to fix timeout errors in DRF applications.

Improved NoSQL Injection support for Assess and Protect.

NoSQLi now handles MongoDB ObjectID types.

Added html.escape as a sanitizer in Assess.

Added WARN level logging if the configuration required to connect to the Contrast Service is missing.

Added INFO level configuration state logging, including ENV and YAML values.

Added YAML validation and, if invalid syntax is detected, WARN level logging indicating such.

Added INFO level application identification logging

Removed strict compiler flags from extension build.

Reduced latency in Django Rest Framework's response handling.

Investigated excessive DB_WRITE propagation.

Inability to patch held references to older versions of modules prevents instrumentation of referenced, rather than directly invoked, methods, such as in Werkzeug version 0.16.0.

Added support for Assess rules:

Removed HTTP request methods as a dataflow source.

Added support for Assess configurations assess.enable_scan_response and assess.rules.disabled_rules.

Failed to repatch module due to __dict__ changing size while iterating over it. (PYT-1085)

Unable to instrument applications on OSX using locally built Python versions due to maxprot setting. (PYT-1025)

Hardcoded analysis rules were accidentally disabled. (PYT-1027)

When the agent was disabled, attempting to start without the Contrast service, resulted in application crash in Flask applications. (PYT-1012)

Instrument compile as part of the Unsafe Code Execution rule.

Decouple ServiceClient from SettingsState

Use normalized_response_headers in DTM instead of response_headers.

Refactored XSS postfilter logic for checking allowed content type.

Updated MongoDB update_methods to account for all arguments.

Replaced FlowMap Technology Analysis.

Verifed that SR handles empty observed route url.

Merge all rules apply_rule into one implementation.

Do not report handled exceptions in INFO/ERROR logs

Upgraded Python agent to use SR 2.11.x.

Fix and update regex used for protect XXE rule (PYT-94)

Fix error in DB write propagator. (PYT-971)

Agent fails to identify itself with new SR instance after the original SR instance goes down. (PYT-715)

List pip and pkg_resources as dependencies and/or include as external modules. (PYT-974)

Do not report observed route if signature is missing/empty. (PYT-970)

Added route coverage support for Django 3.0.

Added Falcon 2.0 support.

Improved accuracy of library file usage.

Improved propagation through regular expressions in Assess.

The team made significant internal cleanup to Request representation

Fixed a bug where regex propagation was throwing an exception under certain conditions.

Fixed a bug related to agent handling of very short JSON keys and values.

Updated protobuf dependency requirement in response to incompatibility issues with older versions.

Fixed an issue where the agent raised an internal exception for applications using certain features of pyasn1.

Fixed a bug where Django applications were unable to properly parse the Content-Type header if a charset was explicitly provided.

Improved error handling around stack trace construction.

Falcon 2.0 is supported and is in beta

Upgraded Contrast Service to 2.8.1

Added support for Django Rest Framework

Added copyright to all agent files

Removed the agent's external dependency on the wrapt package

Improved INFO level logging for easier tracking of applications with multiple processes

When running the agent with protobuf-3.6.1 sometimes the application crashed, which has now been resolved with a newer protobuf version.

Added initial support for Stored XSS rule in Assess for django framework.

Added Unvalidated Redirect support for Assess for pyramid and webob objects.

Made updates to reduce number of false positives from Reflected XSS rule in Assess.

Removed the agent’s external dependency on the six package.

When running the agent under Python 2.7 on Ubuntu 16.10 some instrumentation failed to apply, which has now been resolved.

When applications used str.format in certain edge cases, the agent lost dataflow propagation, which has now been resolved.