Feature/mtconnect/agent - #4318
Conversation
3e509ee to
df19f2f
Compare
|
Moving back to draft pending current (and future feedback). |
7a2952c to
ff16948
Compare
Add a first-class, INI-configurable MTConnect feature: a userspace, non-realtime HAL component that reads machine status, kinematics and tool data via the linuxcnc Python module and publishes them over MTConnect, with no dependency on the external cppagent. Transports (INI [MTCONNECT]TRANSPORT = http, mqtt, shdr): - Embedded HTTP agent: /probe /current /sample /assets, plus /twin and /models. - Standard MTConnect MQTT binding (retained Probe = discovery); optional, python3-paho-mqtt as a Recommends. - SHDR adapter (default port 7878) to feed an external agent such as cppagent, which is configured with a Devices.xml produced by --dump-probe. Device model, generated automatically from [TRAJ]/[KINS]/[AXIS_*]/[JOINT_*]: - Axes as Linear/Rotary components with Motion elements and CoordinateSystems; a compact x:Kinematics extension carries the kins module, coordinates and the joint<->axis map for auto-configuration. - Travel limits and the [SPINDLE_0] speed band as Specifications; work offsets (G54..G59.3 + G92) and tool offsets as WORK_OFFSET/TOOL_OFFSET tables; tool table as CuttingTool assets; coolant flood/mist; XY work rotation. - LINE_NUMBER reports the executing line (stat.motion_line), not interp read-ahead. - Arbitrary HAL pins as data items via [MTCONNECT]HAL_ITEM (hal.get_value). All lengths are emitted in canonical millimetres with nativeUnits. Documents validate against the MTConnect 1.7 schemas; a small LinuxCNC extension schema (mtconnect-linuxcnc-1.xsd) covers the few LinuxCNC-specific items that Streams cannot otherwise represent. Digital twin: an opt-in browser viewer (ENABLE_TWIN) that loads geometry referenced by SolidModel and animates it from /current; MODEL_AUTO generates placeholder geometry from the travel limits. Its three.js comes from the distribution's libjs-three package (served from /three/), so no third-party JavaScript is vendored into LinuxCNC and the twin still works offline. Packaged as a userspace component: src/hal/user_comps/mtconnect-agent.py (USER_COMP_PY -> bin/mtconnect-agent), the mtc package under lib/python, and data assets under share/linuxcnc/mtconnect. Offline test suite in the demo config. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Man page docs/src/man/man1/mtconnect-agent.1.adoc (auto-globbed into the build). - Integrator chapter docs/src/config/mtconnect.adoc, registered in Master_Documentation.adoc and the docs Submakefile. - Document the [MTCONNECT] section in ini-config.adoc. - Demo-config README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Home Assistant's MQTT Discovery is an HA-specific convention, not an open standard, so it is deliberately kept out of the core agent. This optional, opt-in contrib republishes the agent's device model as HA discovery. - configs/sim/axis/mtconnect/contrib/mtconnect-ha-bridge: reuses the agent's device model to publish HA discovery + state, configured entirely with arguments so no HA-specific key is placed in the machine INI. - configs/sim/axis/mtconnect/contrib/ha.py: the discovery/state helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the bundled pure-Python INI fallback from IniReader and always use linuxcnc.ini(). The fallback was a prototype convenience for running offline without the linuxcnc module, but now that the agent ships inside LinuxCNC that module is always available -- and the fallback parsed differently from linuxcnc.ini (it stripped inline '#' comments, whereas LinuxCNC keeps everything after '='), which could mask configuration bugs. IniReader is now a thin typed wrapper over the canonical parser. Raised in PR review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ff16948 to
523fa38
Compare
|
I've removed the vendored threejs in favor of the older debian package. |
|
I'm not sure we should make this a requirement for LinuxCNC. The mtconnect is an addition to the system not really necessary for running a machine. It is actually a very specific addition for a very select group of users. Adding hard dependencies pulls in a lot that, IMO, should be no hard requirement. Don't get me wrong, I think this project certainly has merit, but I'm concerned it only has very narrow application. Certainly something you'd want to choose to add, but a requirement for a normal install is a bit much. At least it should be separated out of the main .deb. I'd like to hear other's opinion on this subject too. |
I think this is needed. Maybe not today, but tomorrow. There have been a number of attempts at http: interfaces etc but none of them are standards based and none of them address security issues satisfactorilly. In fact, I recently proposed a secure solution which a friend James Walker in NZ coded. https://github.com/joco-nz/linuxcnc-Headless-UI MTconnect is standards based and is supported by all of the major manufacturers and Siemens provides native support in their controllers. Being read only, security is not an issue the standard needs to address. Fanuc has been forced to offer a MTconnect adapter despite having its own proprietory communication system. LinuxCNC should not ignore this standard as it could preclude its inclusion in a factory fleet, perhaps where a bespoke machine is needed. I personally know one workshop owner running 7 LinuxCNC machines which are all retrofitted "big iron" machines. He could benefit from data collection and a centralised view of his workshop. At a basic level, Home Assistant via MQTT is seamless and might suffice in that scenario. There have been an increasing number of machine manufacturers adopting LinuxCNC that have a family of machines to address a vertical market, be it a stone manufacturer in China or Wood machinery in India (2 examples known to me) that could benefit from tomorrow's technology if available today. In one case study, an automotive plant avoided downtime that cost $1.7m an hour because their MTconnect data identified increasing torque on a single motor on a CNC axis and could schedule repairs without downtime. Why can't LinuxCNC earn a place on that factory floor? It can't without MTconnect. To me, adding optional MTconnect support via a couple of optional INI parameters is a no brainer and I strongly support this PR. To wave it off as being a very narrow application is simply out of touch with the reality of modern CNC machine applications. We should not be living in the past. |
A lot hinges on the word 'this'. Do you mean digital twins or MTConnect in general? I can take or leave digital twins. They are nice and flashy but not critical. MTConnect as a supported standard, OTOH, is probably more important than you think and will be more important going forward. I've been paying attention to it for five or six years and only recently realized how important it might become for integrating a machine tool into a broader ecosystem of monitoring, CAM, dashboard, and related tools How this is supported is really the crux of the PR and the single most important question. Let me illustrate.... It's the year 2028 and someone googles (or asks chatgpt) "Does LinuxCNC support MTConnect?" Do we want the answer to be ... or B) I wouldn't argue the same way for every feature under the sun. But if the feature is about supporting publicly available, verifiable standards, then I think the development community should strive to make the experience seamless, predictable, and robust. That makes it easy for content creators to make videos. It makes it easy for commercial companies to choose linuxcnc as an embedded control system. And it makes it easy for hobbyists to tinker. |
|
For the record, Sliptonic's pre-release work has already been implemented in the riocore project which supports linuxCNC on unconventional hardware. https://github.com/multigcs/riocore/tree/dev/riocore/plugins/mtconnect |
|
If MTConnect support is what this PR is about, I think the twin and libjs-three could be dropped. Better alternatives already exist: the cppagent repo ships a generic three.js twin (demo/twin, what demo.mtconnect.org/twin runs) that renders any agent from /probe alone, WebXR included, and TrakHound/Ladder99 cover the dashboard side. SolidModel geometry is referenced, never streamed, precisely so external viewers do this job. If a native built-in viewer is wanted, I would rather see something JS-free (a server-side SVG projection of the MODEL_AUTO geometry would be dependency-free and air-gap safe), but I am not putting my foot down on it; if there is a strong argument to keep it as is, I am happy to hear it. Dropping the twin also drops the dependency story to a single, essential, much more palatable item: python3-paho-mqtt, ~100 KB, only needed when the MQTT transport is used. One thing worth considering for the native-vs-separate question: release cycles. In-tree means agreeing to a slow rollout (2.10 is far out, distro packages lag further), while a separate repo with a self-managed apt gets fixes to users in days. The agent has zero core coupling, so both models are viable; just be aware of what you are signing up for either way. |
|
I really like the project and the integration of a simple front end. On Linux in particular, it’s not easy to find something that works straight away, and not everyone fancies setting up a database and Grafana just to have a look at it. However, I’m also of the opinion that it would be better off in its own repository. After all, it’s a standalone component that’s much easier to maintain that way. |
Note we already have user space components that require this but we have not added it as a dependency.
Its not a stand alone component, its part of a modern machine tool's infrastructure. MTconnect is a mature open source standard with a defined scope supported by most major machine manufacturers. This means there will be fewer changes coming than you first think as the initial feature set is very complete. If you leave it for a third party to host, nobody will find it. LinuxCNC should position itself for the future and include these few lines of python code. |
|
This project, as I said before, has merit and do think it to be a good project. But, there are some issues we need to resolve. I think it is a very nice add-on project/component and should be be handled as a separate deb package. I'd even argue that many more parts of LCNC need to be split into separate deb packaged and this project just reminded me to bring that up. That doesn't mean it can't be in the repo, or under LCNC umbrella, but we need to start modularizing. There are questions about the viewer part. Adding all the files from three.js to the repo will become a maintenance nightmare and they aren't even part of the core project. Sure, having the viewer is a nice thing to have, but shouldn't that simply be pluggable on the user's wishes? Also seen in the light that there may be alternatives, a pluggable system would seem better and it would be a separate project. It may be under the same umbrella or separate. Adding MQTT is not the problem. Adding MTConnect is not the problem. |
MTConnect being stable and unchanging is not really the issue here. What we are talking about is the glue between LinuxCNC and MTConnect, and that glue is brand new. I already found a few real issues in it (see my review comments above), and as Bertho pointed out, things change in LinuxCNC that will affect the glue over time. The standard being mature does not make the integration mature. That said, I don't think this "small" python add-on should be split out on its own. There are plenty of optional, narrow-audience comps already in the main deb (mb2hal, mqtt-publisher, the VFD comps), and starting the modularization experiment with the newest arrival would not be consistent. I agree with Bertho that more of LinuxCNC should be modularized, but that is a bigger discussion that should not be settled on the back of this PR. From a user perspective, one install that just works removes a lot of friction; not everyone knows which extra packages they need or how to manage them. The maintenance concern Bertho raises is real, but I think the answer to it is testing rather than packaging. The PR already has an offline test suite, it is just not wired into |
Fixes from the LinuxCNC#4318 review: - http_agent: escape errorCode/message in MTConnectError documents. The message embeds the raw request route, so an unescaped '<'/'&'/'"' broke well-formedness and a crafted route could forge a sibling <Error> element. - shdr_agent: sanitize '|', tab, CR and LF in SHDR values. SHDR has no escaping, so e.g. a program named 'a|b.ngc' corrupted field/line framing. - lcnc_source: read the tool pocket from stat.toolinfo(); tool_table entries carry no pocketno, so the POT Location previously reported the tool number (wrong on random toolchangers). Folds the comment lookup into the same call. - docs: register config/mtconnect.adoc and man1/mtconnect-agent.1.adoc in po4a.cfg so they are not silently dropped from translated builds. - configs: simplify the example .gitignore (drop __pycache__/*.pyc already covered upstream; ignore the whole models/ directory). - test_mtc.py: resolve config paths relative to the test file, not the CWD, so the offline suite can be run from anywhere. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
I made some minor changes to test_mtc.py to resolve config paths relative to the test file, not the CWD so the offline suite can be run from anywhere. This should make wiring into the CI easier if/when that is desired. |
Had a look at https://tracker.debian.org/pkg/three.js, so Debian uses version 111 with version 185 available. The debian/changelog lists some good guys, so I guess it is just not so easy, typically because of extra dependencies drawn in that are not in the distribution. |
|
It is not only about interfacing the softwares. Yeah, I know, we ITlers are trained to think only about technical facts. But please allow to be reminded it is also about interfacing the people behind those softwares. And I suggest to give these friendly&constructive FreeCADlers the warmest-possible welcome and to wholeheartedly embrace them. So, I would risk starting with mtconnect directly on board. From there, once everything becomes a bit more predictable, you can still transform it into a subproject or whatever the consensus may be. |
6 participants
Add native MTConnect support
A userspace, non-realtime HAL component (mtconnect-agent) that exposes LinuxCNC machine status, a full kinematic description, and tool data over the MTConnect standard — no external cppagent required. Configured from a small [MTCONNECT] INI section; the device model is generated automatically from [TRAJ]/[KINS]/[AXIS_]/[JOINT_].
Packaged as a userspace comp (bin/mtconnect-agent, lib/python/mtc, share/linuxcnc/mtconnect) with a man page, an integrator chapter, and an offline test suite in the demo config (configs/sim/axis/mtconnect).
Three commits: native agent, documentation, optional Home Assistant contrib (kept out of core since HA discovery is HA-specific). MQTT needs python3-paho-mqtt (added as a Recommends).