Actions from last time
- DONE: Lars will reach out to Hillel Wayne once more during this iteration.
- Lars emailed, we got a response, to be discussed in this meeting.
Assessment of the iteration that has ended
We extended, by mutual agreement on Matrix, to extend this iteration by two weeks. We’re both a little short on time and energy for Subplot, lately.
We had chosen the following issues for this iteration:
- web#3 – Document how we do meetings
- assigned to Daniel
- subplot#283 – YAML metadata is not in its own file
- assigned to Lars
- subplot#286 – Tests sometimes fail
- assigned to Lars
Unfortunately, despite the extension, we didn’t make much progress on these issues. Rather than extend the iteration again, we’re having a meeting.
We decided to extend the iteration again, by another two weeks, rather than close this iteration and create a new one with the same issues.
Discussion
Subplot development speed
We discussed whether the slow development speed should worry us, and what we can do to make at least some concrete progress each iteration.
We decided that we don’t need to worry, as such, but that there are things we can do to make sure some progress is made in each iteration. Most importantly, we should include at least some tasks that are small enough and that we understand well and are enthusiastic about. We tentatively set the limit on the estimated effort to one hour. Tasks bigger than that should have at least one sub-task carved out that’s smaller.
Release to benefit Sequoia?
We discussed making a release of Subplot to help the Sequoia project. It would mean Subplot doesn’t depend on the clap
version 2 anymore, and that would help Sequoia reduce its dependency tree and build times.
There have been nothing very breaking since the previous release. We decided to make a release soon, and call it 0.4.2. We created subplot#289 to track this, and assigned it to Lars.
On YAML metadata
Lars has attempted to unify the two document metadata representations we have, but feels there’s too much entanglement with the older one to allow for an easy refactoring. This is the subplot#283 issue.
He now feels a better approach may be to convert docgen
into using the pulldown-cmark
parser for Markdown instead, and has made some progress on that. The current problem is that the rudimentary code to build a Pandoc AST from pulldown-cmark
parsing events is meant only for codegen
, and is highly insufficient for docgen
.
We discussed whether to use the Pandoc Abstract Syntax Tree representation, replace that with an HTML abstraction, or work on the pulldocn-cmark
parsing event stream directly. Lars will experiment. Daniel noted that the event stream can contain source information, which might make the marked-markdown work unnecessary.
Feedback from Hillel Wayne
We got feedback from Hillel Wayne, who is a well-known authority on formal methods for software correctness. He made good points. Our takeaways:
- Our website and other documentation needs to be more oriented towards those not involved in developing Subplot.
- we can achieve part of this by re-arranging existing content
- we could write new content, such as a “Subplot book”, a la similar documentation efforts for Rustup, Cargo, BuildStream, and other projects
- Our presentation of what Subplot is needs clarification. Subplot is tooling around Gherkin, or a variant of it, but we may not actually say that or say it clearly.
- We do think it’s possible to write acceptance tests in Gherkin. We should expand on what we understand acceptance testing to be.
- We should improve our examples. The current ones are rather simplistic. We should add explanatory text to the examples to make it clearer what the purpose of each example is.
- We should write documentation to describe how we expect Subplot to be used, when used well: who should write subplot documents, who should consume them, how they should be written to be good, how to design scenario steps, etc.
- we should also discuss the layered approach described by Daniel in his FOSDEM’22 talk about using Subplot for safety
Daniel will draft a response and Lars will review it.
The gitlab.com hosting situation
We reviewed the billing page for Subplot on gitlab.com. It all looks OK. We’re over the limit on seats, but it seems we can increase the seats for free.
We will review the billing page every iteration or two.
Packaging and dependencies
We discussed what we can do to make Subplot easier for other projects to use Subplot, or to package Subplot, especially for Linux distributions like Debian.
When other projects use Subplot, it shouldn’t require pulling in a large set of dependencies, whether they are other tools such as Pandoc and LaTeX, or Rust crates. The bigger the set of unavoidable dependencies is, the more costly it is to use Subplot, in terms of computing resources, and maintenance time and effort in the long run.
Debian aims to be self-contained: to build everything in Debian may only require software packaged in Debian and must not require downloading anything from the Internet, outside the Debian package archive. This helps Debian be more stable, less vulnerable to changes in tens of thousands of other projects. The more dependencies Subplot has, the more work it is to package.
We should thus keep the dependency tree of Subplot small, when we can do so with acceptable sacrifices on our part.
- Each Subplot crate is aimed at a different use case. We should ensure it dependency tree doesn’t accidentally pull in unnecessary crates, or extra versions of crates.
- We should aim to keep crate dependencies for building, developing, and testing as tidy as possible. Additionally, we should investigate how we can tidy up dependencies that are only needed for producing binaries, rather than using a crate as a library.
Repository review
We reviewed issues updated since the previous meeting, merge requests, and CI pipelines for all the projects in the Subplot group on gitlab.com.
Subplot Container Images
- Open issues: 1
- Merge requests: 0
- Additional branches: 0
- CI: OK, ran on Friday
subplot-web
- Open issues: 2
- Merge requests: 1
- Additional branches: 0
- CI: Ick, OK.
subplot
- Open issues: 48, newly changed ones reviewed
- subplot#287 – Needs documentation for Rust step functions for implementing contexts
- assigned to Daniel
- subplot#288 – Should an on-disk filename be captured1 as a Path?
- added to current goal
- subplot#289 – Make release 0.4.2
- assigned to Lars
- subplot#287 – Needs documentation for Rust step functions for implementing contexts
- Merge requests: 1
- Additional branches:
docgen-cmark
, Lars is keeping this around for reference for future changes
- CI: OK
Recurring items issues
There are some issues tagged recurring-agenda-item
in the issue tracker, which we discuss in every iteration meeting.
- subplot#119 – Subplot is in not in Debian unstable
- we will work on reducing Subplot dependencies that aren’t already in Debian
- subplot#272 – Migrate project to issue-based iteration meetings
- we will try a “merge request based approach” instead, for a few iterations
- Lars has tried this in the Obnam project and it has worked OK
- [[!subplotissue 273[[ – Perform whole-codebase review and cleanup
- we are not up to doing this yet
Goals
Goal 3: Current goal
Subplot is altered to treat a YAML document as the root of any subplot document, permitting multiple markdown files to be joined into a single scenario set or output document. We remove our dependency on Pandoc’s AST for internal document representation, and support a better range of document metadata as a result. We will use marked data types where possible to permit better provenance information in error and warning messages; and we will strive to reenable the reference tests such that we can make a statement to the effect that post goal 3, we do not intend to make further breaking changes to the input formats or primary behaviours of Subplot.
Goal 4: Planned
This goal is very likely to change. This is merely an initial sketch.
We’ll improve documentation and generally aim at making Subplot more plausibly usable by people outside of the Subplot project.
Plan for new iteration
We intend to work on the following issues:
- web#3 – Document how we do meetings
- assigned to Daniel
- web#4 – Review FAQ
- assigned to Daniel, Lars will help as needed
- subplot#283 – YAML metadata is not in its own file
- assigned to Lars
- subplot#286 – Tests sometimes fail
- assigned to Lars
- we’ve only seen this fail on Lars’s Ick CI
- Lars will try running the build many times to get an idea of how often the tests fail
- subplot#287 – Needs documentation for Rust step functions for implementing contexts
- assigned to Daniel
- subplot#289 – Make release 0.4.2
- assigned to Lars
Actions
These are in addition to work on issues chosen for this iteration.
- Daniel to draft a response to Hillel Wayne. Lars will review.
- Daniel to research how to have dependencies only for binaries produced by a crate, but not the library part of the crate.
- Lars to upgrade his
roadmap
crate to useclap
version 3, instead ofstructopt
.
Meeting participants
- Daniel Silverstone
- Lars Wirzenius