Generate list of unimplemented exercises for a track #149

thriqon · 2018-10-14T19:06:57Z

fixes #144.

nywilken · 2018-10-15T03:06:06Z

@thriqon thanks for submitting a PR to address #144. I’ll take a look and get back with some feedback during the week.

fixes #144.

kytrinyx

I took a quick look at this conceptually (I will take a look at the code side of things in a separate pass).

Conceptually I want to make sure that we're actually defining/testing all the cases that we need (over the years, exercise definitions have grown quite complex!).

I think the use cases that we need to cover are:

happy path 1: the problem specification exists (present, valid, not deprecated), and the exercise is implemented in the track
happy path 2: the problem specification does not exist, and the exercise is implemented in the track with custom metadata in .meta
missing specification: the problem specification does not exist, there is no custom specification in .meta, but the track has an implementation. The message should be that there is no specification.

This last thing might mean any number of things:

is there a typo in the exercise directory name?
is this a completely new exercise? If so, should it be submitted as a new, generic exercise to Exercism's problem specifications? Or should this be a custom exercise just for this track? I'm not sure what the error message should be in this case; maybe just "no specification found", and we can figure out documentation later.

Then we have the deprecation stories (assume the exercise exists both in the problem specification repo and the track repo).

the problem specification is deprecated, and the exercise is deprecated. No message. (The curriculum team has decided that the exercise is not a good one, and the track maintainers have done the work to deprecate it. It's all good.)
the problem specification is not deprecated, the exercise is deprecated. No message. (The problem specification is fine, but the track maintainers have decided it's not a good fit for the track. It's all good).
the problem specification is deprecated, the exercise is not deprecated. Needs a message. In this case we think this exercise is not a good fit for the platform, and the maintainers should consider deprecating it.

Then there is the foregone story, which you've covered: we have an implementation, but the track maintainers said this is not a good fit for the track, so they labeled it as foregone.

I think that's all the use cases we have at the moment. Eventually there may be something about versioning, where the canonical data has been updated but the implementation in the track doesn't reflect the new changes, but I don't think we've actually formalized any metadata around how the tracks keep track of this.

Update: It looks like there are primitives for keeping track of versions based on the canonical data. Can you confirm how these are specified within the track implementation? What happens if the implementation in the track doesn't specify a version at all?

kytrinyx · 2018-11-02T14:29:43Z

...s/implementation-status/problem-specifications/exercises/not-implemented/canonical-data.json

+{
+	"exercise": "not-implemented",
+	"version": "1.0.0"
+}


Is this how not-implemented exercises are represented? I'm thinking, let's say that the Python track decided to implement an exercise apple-pie. If the problem specifications repository has an apple-pie directory, I would expect the exercise to be implemented. It might not have canonical data (but that's maybe fine; I think the canonical data is optional).

Another use case would be that there is an apple-pi exercise (some sort of play on the mathematical concept of "pi"), but then a track tries to implement apple-pie (type). It would not find a directory with that name in the problem specifications. Would it be correctly reported as not implemented?

kytrinyx · 2018-11-02T14:30:22Z

fixtures/implementation-status/problem-specifications/exercises/up-to-date/config.json

+{
+	"exercise": "up-to-date",
+	"version": "2.0.0"
+}


Do exercises in the problem specifications repo have a config.json file?

kytrinyx

Ok! I took a look at the code. I think we need to make sure we're actually sure what we're trying to do here and all the various cases we're checking before going into the implementation too much!

kytrinyx · 2018-11-02T14:45:17Z

cmd/implementation_status.go

+	"github.com/spf13/cobra"
+)
+
+var pathExampleProblemSpecifications = "<path/to/problemspecifications>"


Nitpick: I think it would be worth calling the directory problem-specifications here, since that is what the repository is named. If people clone it directly, then that is what their directory name will be called.

kytrinyx · 2018-11-02T15:11:05Z

cmd/implementation_status.go

+
+For example, if upstream prepares a new exercise it is important to either
+implement it in a track or declare it foregone. If a new version is published the update
+should be made available soon.


Rather than say "for example", let's clarify the ways in which the implementation would typically need attention.

Also, I think it's important to consider whether the exercise is a good fit for the track, but not necessarily to implement it.

So maybe something like:

When a new exercise is added to the problem specifications repository it is available to be implemented in the language tracks.
When a problem specification is deprecated, track maintainers should consider deprecating the corresponding implementation.
When the canonical data of a problem specification is updated, the corresponding implementation may need to be updated.

I wonder what we will do in the case where the maintainer decides that they don't want to make the suggested change. I think we should think this through before we finalize this functionality, as this would probably break the builds without giving people a way out.

kytrinyx · 2018-11-02T15:13:48Z

cmd/implementation_status.go

+
+	Args: cobra.ExactArgs(2),
+
+	Example: fmt.Sprintf("  %s implementation-status %s %s", binaryName, pathExampleProblemSpecifications, pathExample),


Let's give the problem specification path a default value. My guess is that most people would stick it in a directory parallel to the track directory:

. ├── problem-specifications └── the-track

That way if they are using the default they don't need to actually say anything here.

Another thing to consider is how this command would be run on Travis. Will we need to clone the problem-specifications repository to run it? (Probably). Do we already do this? (Maybe). What would the command look like in that case? (No idea!)

kytrinyx · 2018-11-02T15:20:50Z

cmd/implementation_status.go

+)
+
+// listProblemSpecs gives a map of exercise name and whether the exercise is deprecated.
+func listProblemSpecs(path string) (map[string]int, error) {


If problem specifications have both deprecation status and version information, we may want to make this an array of a custom struct type.

That would help with the clarity of the function name as well. The name list is a bit ambiguous (a list is not a map). I think a map is right, but maybe a map of slug to problem specification (an actual struct value with a deprecation field).

kytrinyx · 2018-11-02T15:23:13Z

cmd/implementation_status.go

+			ret[f.Name()] = implementationStatusOK
+		} else {
+			ret[f.Name()] = implementationStatusDeprecated
+		}


We need to handle the case where it's an error, but not an os.IsNotExist error.

kytrinyx · 2018-11-02T15:30:23Z

cmd/implementation_status.go

+}
+
+func trackConfigExercises(track string) (map[string]int, error) {
+	var trackConfig struct {


We read the track config elsewhere in this program, right? Should this be a package type?

kytrinyx · 2018-11-02T15:32:48Z

cmd/implementation_status.go

+	return ret, nil
+}
+
+func trackConfigExercises(track string) (map[string]int, error) {


Here, too, the name is pretty ambiguous. I wonder if a map of name to status is a bit too cryptic here. Maybe we need a type for this, too. Or maybe the name should just be something like statusByExerciseSlug to clarify the idea that it's a map of statuses.

kytrinyx · 2018-11-02T15:37:13Z

cmd/implementation_status.go

+const (
+	implementationStatusOK = iota
+	implementationStatusDeprecated
+	implementationStatusForegone


I noticed in the comment for the command that it talks about versions. Is this being implemented?

Smarticles101 · 2019-01-24T18:54:24Z

Is this still being worked on? If not, I'd be thrilled to pick up where it was left off and polish it up for merging.

rpottsoh · 2019-01-24T18:56:51Z

You've got my vote. 😄

thriqon · 2019-01-24T19:33:04Z

Mine too! Sorry for letting this slide, I'm currently unable to find time for this... thanks @Smarticles101 for picking up.

Smarticles101 · 2019-01-24T19:39:38Z

@thriqon no problem! I'll open a new pull request and build off your existing code. :)

Generate list of unimplemented exercises for a track

7e58c7b

fixes #144.

kytrinyx suggested changes Nov 2, 2018

View reviewed changes

Smarticles101 added a commit to Smarticles101/configlet that referenced this pull request Feb 28, 2019

Cleaning up exercism#149

eb1717e

Smarticles101 mentioned this pull request Feb 28, 2019

Generate list of unimplemented exercises for a track #155

Open

Smarticles101 added a commit to Smarticles101/configlet that referenced this pull request Feb 28, 2019

Cleaning up exercism#149

5aa30dd

Smarticles101 added a commit to Smarticles101/configlet that referenced this pull request Feb 28, 2019

Cleaning up exercism#149

4434a4c

Base automatically changed from master to main January 28, 2021 22:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate list of unimplemented exercises for a track #149

Generate list of unimplemented exercises for a track #149

thriqon commented Oct 14, 2018

nywilken commented Oct 15, 2018

kytrinyx left a comment •

edited

Loading

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx left a comment

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

kytrinyx Nov 2, 2018

Smarticles101 commented Jan 24, 2019

rpottsoh commented Jan 24, 2019

thriqon commented Jan 24, 2019

Smarticles101 commented Jan 24, 2019


		Args: cobra.ExactArgs(2),

		Example: fmt.Sprintf(" %s implementation-status %s %s", binaryName, pathExampleProblemSpecifications, pathExample),

Generate list of unimplemented exercises for a track #149

Are you sure you want to change the base?

Generate list of unimplemented exercises for a track #149

Conversation

thriqon commented Oct 14, 2018

nywilken commented Oct 15, 2018

kytrinyx left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kytrinyx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Smarticles101 commented Jan 24, 2019

rpottsoh commented Jan 24, 2019

thriqon commented Jan 24, 2019

Smarticles101 commented Jan 24, 2019

kytrinyx left a comment •

edited

Loading