Subplot → blog → 2021 → 10 → 23 → Iteration planning: October 23rd
  1. Review of actions from last meeting
  2. Review of the iteration that has ended
  3. Review of the repositories
  4. Current goal (goal 2; not changed for this iteration)
  5. Issue review
  6. Other discussion
    1. Reworking for goal 3
  7. Plan for next iteration
  8. Other business
  9. AOB
  10. Actions

Review of actions from last meeting

Actions from last time, and their status are:

  • Lars to create a label for blocking-high-profile-projects (name to be decided by Lars when he does it) and to label at least the Sequoia-PGP blockers if not also anything else he thinks may be blocking projects such as Rustup. Half done, carrying

  • Lars to record decision about MSRV in DECISIONS.md in the subplot repository. DONE

  • Lars and/or Daniel to add a Docker image for CI with Rust 1.48.0 installed, or add it to the main image. DONE

  • Lars and/or Daniel to change .gitlab-ci.yml to build and test using the 1.48.0 toolchain. DONE

  • Lars and Daniel to discuss whether to change ./check to take an option to specify which toolchain to use or to keep that outside of the script. DONE (We selected external control)

  • Lars to refactor the changes for build.rs support, as discussed, and push an MR for it. DONE

  • Lars should file an issue about the get_basedir_from changes, if the problem can be reproduced with the original code. No reproduction, no issue filed

  • Lars to document decision that it’s the user’s responsibility to set up the environment in which tests are run. DONE

Review of the iteration that has ended

Subplot milestone 37 had the following issued closed:

Some progress was made on:

The following issues remain open:

We moved the non-release-process remaining issues to the next milestone (Subplot milestone 38) and then closed off this iteration.

We removed Subplot issue 207 and Subplot issue 218 from the future iterations, and removed Subplot issue 207 from the current goal.

Review of the repositories

There are no MRs currently open in subplot, subplot-web, or subplot-container-images.

The subplot-container-images and subplot-web repositories do not extra branches. The subplot repository has extra branches:

  • subplot-rust – Daniel is keeping this around until he has completed the work on Subplot issue 198.
  • polyglot-docs – Daniel is working on Subplot issue 198 here.
  • docgen-cmark – Lars is working on making docgen use cmark rather than pandoc for the initial parse and processing.

We triggered the pipeline to build the container image for Subplot. All subplot pipelines are clean.

Current goal (goal 2; not changed for this iteration)

Subplot provides a set of libraries with identical capabilities in each of the supported languages. Python remains a supported language. Rust is promoted to supported-language status. Subplot will be tested with all supported languages. In addition, any quality of life improvements which can be done shall be done. This goal will be considered complete when a release of Subplot has been made with the unified language handling support complete.

Issue review

We did a full review of all non-someday-maybe issues.

Changes we made include:

  • Subplot issue 126 – Create a CI cronjob which attempts to cargo update everything We opted to close this issue as we think the current release process is sufficient for now.
  • Subplot issue 133 – No feature table comparing scenario runners from different codegen templates We opted to mark this issue as blocked by Subplot issue 198 (multi-impl-lang docs)
  • Subplot issue 141 – subplotlib: Needs better test suite We opted to mark this as blocked by Subplot issue 198 and labelled it as tracking since we’ll close it when the same subplot documents are testing runcmd, files, etc. for all language templates.
  • Subplot issue 180 – Doesn’t warn about unused embedded files We agreed we should now be able to lint this as part of codegen so tentatively assigned it to Lars for this iteration.
  • Subplot issue 188 – Old Subplot libraries have bindings with regex that could now be simple patterns We tentatively assigned this to Lars for this iteration.
  • Subplot issue 213 – _“Failure: document has no title” when there’s duplicate keys" We are going to defer this to the new goal-3 as per below discussion notes
  • Subplot issue 214 – Lacks examples and docs for using Rust step functions We marked this as blocked by Subplot issue 198
  • Subplot issue 220 – Generated Rust test programs are hard to debug We marked this as blocked by Subplot issue 198
  • Subplot issue 221 – Command line tool as insufficient logging We made a note on the issue to the effect of the next steps. We added this to the current goal, but not to the iteration.
  • Subplot issue 222 – Typeset output could indicate snippets are a continuation of the scenario and not a new scenario We added the typesetting label to this and also someday-maybe.
  • Subplot issue 223 – lib/runcmd lacks a generic “given an installed {name}” step We added a blocked-by on Subplot issue 198. We still need further discussion on this.
  • Subplot issue 224 – Logging feature in the subplot binary needs tests We updated the title to better indicate that this is about testing the behaviour of the SUBPLOT_LOG* environment variables. We also labelled it for goal 3.
  • Subplot issue 229 – Licensing of Subplot libraries and subplotlib needs to be clarified We agreed on MIT, Daniel removed the actively-discussed label. Daniel will take on the task of reaching out (and subsequently changing the licence terms) and so we assigned it to Daniel for this iteration.
  • Subplot issue 238 – Doesn’t support abstract in document metadata Unassigned from Lars for now.
  • Subplot issue 239 – In metadata, author is a single string Unassigned from Lars for now.
  • Subplot issue 240 – Could we mangle generated scenario function names in the Rust template? We decided that the suggestions made on-issue were sufficient and closed the issue
  • Subplot issue 241 and Subplot issue 242 – Related to checking subplot, we made the two issues be linked. We assigned 242 to Lars for this iteration.

We left the docker-for-subplot issue alone, and subplot-web had no issues.

Other discussion

Reworking for goal 3

After significant discussion we agreed that it would be good to replace the current approach of metadata-in-document with a separated metadata YAML file. Essentially that file would be the input to Subplot, and it would list, among other things, the set of markdown files which comprise the scenario input.

Doing this would allow us to have a strict parsing of Subplot specific metadata, but a more liberal parsing of optional Pandoc-specific metadata. The docgen process would then include conversion and aggregation of this metadata into the AST passed to Pandoc.

To this end, we decided that we should defer things currently labelled as goal 3 until a putative goal 4, and we should define goal 3 as the above work. We recorded some actions (see below) to this effect. We hope that this will be the last major breaking change, but we cannot make that guarantee.

One potentially beneficial side-effect of this work is that you will be able to be confident of the action of running Pandoc against any individual markdown file; though if you opted to include Pandoc specific metadata in one of them, the result after docgen would be quite confusing because we intend to use the pulldown_cmark crate for all Markdown parsing and it has no provision for YAML metadata blocks. Part of this goal would be to simplify our metadata handling so we would not want to try and support this in-place metadata at all.

Plan for next iteration

We opened Subplot milestone 38 to cover this iteration, with the following issues:

Other business

  • We only just made a release, so no release planned for this iteration.

  • We are not yet ready to file an RFP bug to get Subplot packaged for Debian. It will happen after we think we won’t be making breaking changes anymore.

  • We are not yet ready to make a whole code base review of Subplot. However we feel at least part of the new goal 3 will be reviewing a portion of the code.

  • We will switch an issue based agenda when other people join the meeting.

  • We relabelled our “reachout set for goal 2” to be for our new goal 3 and we agreed to not reach out after this goal because our new goal 3 is such a large break in API.

  • Regarding a talk at FOSDEM, we have agreed that we want to give a talk, though we’re not certain if it’ll be at FOSDEM and we would want it to be suitable to give at other places as well. Given that we intend to set up a shared space to brainstorm the audience, and the intended knowledge that audience will gain from the talk.

  • We agreed to continue with merge commits, and noted git log --graph as helpful, along with git log --merges. Once either someone queries this decision, or we have more project policy to document, we will write this down either in CONTRIBUTING.md or DECISIONS.md.

AOB

  • Regarding the media repository question - we believe this is premature at this point - once we are ready to write the talk mentioned above, it will be time for the repository.

  • We are not going to do a release in this iteration anyway, so Lars suggests that whoever does the next release plays with cargo release since he has checked and its README suggests that it should support workspaces.

  • We continue to believe that in part, helping high profile projects such as Sequoia-PGP or Rustup remains a big part of helping to convince people to use Subplot. Since we are about to make some more large breaking changes we do not want to convince more people into this camp yet. Writing the talk will be another way to increase visiblility of the project. More “real” examples may also be a good way to show the capabilities of Subplot. All of these things will be a lot of work, so we should be careful to not burn ourselves out on this.

Actions

  • Lars (and Daniel) to label at least the Sequoia-PGP blockers if not also anything else he thinks may be blocking projects such as Rustup.

  • Daniel to shuffle goal3 to goal4 and create goal3 label for the purpose of tracking the work we discussed around redoing metadata as a separate file.

  • Daniel to file a tracking issue for the metadata work, and assign it to Lars. Lars to then break down the work at some future point.

  • Lars to set up a space for brainstorming the talk we want to give.

  • Lars to file an issue with details of cargo audit and cargo deny and the intention to have a CI pipeline step which runs them. This may include filing an issue on the docker container to add the tooling.