WorkflowLearner

By u3588064 GitHub

To teach LLM Agents understand workflows with PSR.exe's hmt.

workflowlearner llm

Overview

What is WorkflowLearner?

WorkflowLearner is a project designed to teach large language models (LLMs) how to understand workflows by utilizing MHT files recorded with PSR.exe, a built-in Windows tool that captures user actions.

How to use WorkflowLearner?

To use WorkflowLearner, first record your actions using PSR.exe, which generates an MHT file. Then, parse the MHT file to extract workflow information and use the provided scripts to train the LLM on this data.

Key features of WorkflowLearner?

Records user operations and generates MHT files using PSR.exe.
Parses MHT files to extract detailed workflow information.
Utilizes LLMs to learn and understand user workflows effectively.

Use cases of WorkflowLearner?

Automating the understanding of user workflows for software applications.
Enhancing LLM capabilities in task automation and user assistance.
Analyzing user behavior for better software design and user experience.

FAQ from WorkflowLearner?

What is PSR.exe?

PSR.exe is the Problem Steps Recorder in Windows that records user actions and generates MHT files.

Is WorkflowLearner compatible with all versions of Windows?

Yes, WorkflowLearner works with any version of Windows that supports PSR.exe.

How can I contribute to WorkflowLearner?

Contributions are welcome! You can submit issues or pull requests on the project's GitHub page.

Content

# 工作流录制通 WorkflowLearner

项目概述 Summary

本项目旨在通过使用PSR.exe录制的MHT文件，让大语言模型（LLM）学习工作流。PSR.exe是Windows操作系统中的问题步骤记录器，可以记录用户操作并生成MHT文件。通过解析这些MHT文件，LLM可以学习和理解用户的工作流程。

To teach LLM Agents how to work with PSR.exe's MHT files, this project aims to use MHT files recorded by PSR.exe to enable large language models (LLMs) to learn workflows. PSR.exe is the Problem Steps Recorder in the Windows operating system, which can record user operations and generate MHT files. By parsing these MHT files, LLMs can learn and understand user workflows.

在线演示demo： https://deepnote.com/app/skedge-d0bd/-9355fadf-b8e7-4dfb-acf1-7c56103d1360?utm\_source=app-settings&utm\_medium=product-shared-content&utm\_campaign=data-app&utm\_content=9355fadf-b8e7-4dfb-acf1-7c56103d1360

功能 Functions

使用PSR.exe录制用户操作并生成MHT文件。
解析MHT文件并提取工作流信息。
使用LLM学习和理解工作流。
Record user operations using PSR.exe and generate MHT files.
Parse MHT files and extract workflow information.
Use LLMs to learn and understand workflows.

安装 installment

前提条件 Requirements

Windows操作系统 Windows operating system
Python 3.x
PSR.exe（问题步骤记录器，Windows自带;Problem Steps Recorder, included with Windows）

安装步骤 Install Steps

克隆仓库/Clone the repository:

```bash git clone https://github.com/u3588064/WorkflowLearner cd llm-workflow-learning ```

安装依赖/Install dependencies:

```bash pip install -r requirements.txt ```

使用方法 Methods

使用PSR.exe录制用户操作 Recording actions with PSR.exe

打开PSR.exe（问题步骤记录器），可通过Win7/Win8/Win10的搜索栏搜索找到。
点击“开始记录”按钮，然后开始你的工作。PSR.exe会自动记录鼠标动作（如点击、拖动、滚动等）以及键盘动作（仅是动作，不会记录键入的内容）。
完成操作后，点击“停止记录”按钮。软件会自动弹出保存窗口。
结果会以一个Zip压缩包存在，压缩包内是MHT格式的网页，内含分步步骤以及屏幕截图。

Open PSR.exe (Problem Steps Recorder), which can be found via the search bar in Win7/Win8/Win10.
Click the "Start Record" button and begin your work. PSR.exe will automatically record mouse actions (such as clicks, drags, scrolls) and keyboard actions (only actions, not the content typed).
Once done, click the "Stop Record" button. The software will automatically prompt a save window.
The result will be saved as a Zip file containing an MHT file, which includes step-by-step actions and screenshots.

解析MHT文件 Parse MHT files:

```bash python parse_mht.py path/to/your/file.mht ```

使用LLM学习工作流 Use LLM to learn workflows:

```bash python learn_workflow.py path/to/parsed/data.json ```

文件结构 File Structure:

``` llm-workflow-learning/ │ ├── parse_mht.py # 解析MHT文件的脚本 ├── learn_workflow.py # 使用LLM学习工作流的脚本 ├── requirements.txt # 项目依赖 ├── README.md # 项目说明文档 └── data/ # 存放解析后的数据文件 ```

``` llm-workflow-learning/ │ ├── parse_mht.py # Script to parse MHT files ├── learn_workflow.py # Script to use LLM to learn workflows ├── requirements.txt # Project dependencies ├── README.md # Project documentation └── data/ # Directory to store parsed data files ```