Skip to content

Chapter: Background - #2

Open
ysndr wants to merge 72 commits into
mainfrom
chapter/background
Open

Chapter: Background#2
ysndr wants to merge 72 commits into
mainfrom
chapter/background

Conversation

@ysndr

@ysndr ysndr commented Dec 27, 2021

Copy link
Copy Markdown
Owner
  • Nickel
    • type system
    • row types
    • optional typing
  • AST
  • LSP Methods

@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

1 similar comment
@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

Comment thread chapter/background.md


## Data oriented languages

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about transformations/term level handling of contracts and recursinve records etc.
Especially contracts as they get rewritten into expressions run during runtime.
I think this concept applies better to the background as I dont use it in the LSP, yet it is quite technical/internal.

Comment thread chapter/background.md
### Software defined Networks
### Software Networks


Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should i go into detail of things like parsing and AST construction here as well?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I imagine you have to say somewhere the following: traditionally, programs are parsed as an AST (explaining quickly what it is). However, for the LSP, we need to enrich this data structure (while not polluting the core nickel implementation) and we need to index things by positions. Those two constraints explain your technical choices.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think i will motivate the need for enriching the AST (and in short describing what is the AST) in #4 or otherwise move the Nickel AST section from Method to Background entirely

@ysndr
ysndr force-pushed the chapter/background branch from b32e9c0 to 70e2aec Compare December 28, 2021 19:01
@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

1 similar comment
@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

Comment thread chapter/background.md Outdated

## Language Server Protocol

### Rationale

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discuss use-cases as part of related work (on other usecases such as dsl's)

Comment thread chapter/background.md Outdated

### Rationale

Since its release, the LSP has grown to be supported by a multitude of languages and editors[@langservers @lsp-website].

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General question: footnotes or refernces for websites

@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

5 similar comments
@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

github-actions Bot commented Jan 1, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

Comment thread chapter/background.md Outdated
Comment thread chapter/background.md Outdated
Comment on lines +55 to +69
`"jsonrpc" : "2.0"`
~ A fixed value format indicator

`"method"`
~ The name of the procedure called on the server
~ May not start with `rpc.` which is an indicator for internal messages

`"params"`
~ An optional set of parameters passed to the executed method.
~ Parameters can be passed as a list of arguments or as a named dictionary.

`"id"`
~ A (unique) identifier for the current message
~ Used to answer client requests
~ Messages without an `id` are considered to be *Notifications*

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this key specific documentation and the following prosa are somewhat redundant.
I'm not sure which to keep, especially considering my comment on documenting the LSP

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can get rid of this indeed. You can link to the LSP official documentation somewhere. Explaining a few things like below is fine, but reproducing technical documentation is a bit superfluous.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think about turning the schema into a short example, just to have an idea at least
Something like

{ "jsonrpc": "2.0", "method": "add" , "params": [1,2] , "id": 12 }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, short examples are always welcome

@github-actions

github-actions Bot commented Jan 1, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

3 similar comments
@github-actions

github-actions Bot commented Jan 1, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

github-actions Bot commented Jan 1, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

github-actions Bot commented Jan 2, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

Comment thread chapter/background.md Outdated

## Configuration programming languages

Nickel [@nickel], the language targeted by the language server detailed in this thesis, defines itself as "configuration language" used to automize the generation of static configuration files.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section describes the use-case for text based configuration files, their shortcomings in terms of validation and language features.
I then mention using general purpose languages for configuration, closing with why one would want to use configuration programming languages

Comment thread chapter/background.md Outdated
Comment on lines +142 to +99
Despite this, not all languages serve as a configuration language, e.g. compiled languages and some domains require language agnostic formats.
For particularly complex products, both language independence and advanced features are desirable.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention the security aspects of using an evaluated language with access to the system?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can mention it. Most of the config languages I know and you cite are pure, that is they can't perform side effects like interacting with the system. It's an important property for both reproducibility (avoid having the configuration unseemly depends on external state) and security (you don't want your config to be able to run arbitrary programs).

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secure also in terms of leaving critical holes in a network which could solve statically checked or enforced against using contracts

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also Chef uses ruby ...
And in the end this would be an argument for DSL like languages with certain restrictions

@github-actions

github-actions Bot commented Jan 2, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

Comment thread chapter/background.md Outdated
Comment thread chapter/background.md Outdated
Comment thread chapter/background.md Outdated
Comment thread chapter/background.md Outdated
Comment thread chapter/background.md Outdated
Language servers are today's standard of integrating support for programming languages into code editors.
Initially developed by Microsoft for the use with their polyglot editor Visual Studio Code^[https://code.visualstudio.com/] before being released to the public in 2016 by Microsoft, RedHat and Codeenvy, the LSP decouples language analysis and provision of IDE-like features from the environment used to write.
Developed under open source license on GitHub^[https://github.com/microsoft/language-server-protocol/], it allows developers of editors and languages to work independently on the support for new languages.
If supported by both server and client, the LSP now supports more than 24 language features^[https://microsoft.github.io/language-server-protocol/specifications/specification-current/] including code completion, hover information, resolution of type and variable definitions, controlling document highlighting, formatting and more.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"If supported by both server and client": I don't understand the logical relation between this beginning and the rest of the sentence.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, implementing language features on the server side does not mean these are available on your editor and vice versa.
In fact both declare sets capabilities.
All 24 features are only available for a combination of editors (clients) and servers that support them all

Comment thread chapter/background.md Outdated
Applications of configuration languages are ubiquitous especially in the vicinity of software development. While XML and JSON are often used by package managers [@npm, @maven, @composer], YAML is a popular choice for complex configurations such as CI/CD pipelines [@travis, @ghaction, @gitlab-runner] or machine configurations in software defined networks such as Kubernetes and docker compose.

Such static formats are used due to some significant advantages compared to other formats.
Most strikingly, the textual representation allows inspection of a configuration without the need of a separate tool but a text editor and be version controlled using VCS software like Git.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You explain why text configuration is better than some random proprietary format + a GUI application to control it. I would also insist in this paragraph on why configuration has become so ubiquitous recently: infrastructure as code and related paradigms. That is, configuration is replacing a lot of things that were done in an imperative manner formerly. Sysadmins 20 years ago were mostly linux users typing commands on servers. Nowadays they are mostly SRE/DevOps writing yaml and deploying through things Kubernetes/docker/Terraform. I don't think you would use Nickel for an npm package description (it would probably be overkill, and here simplicity is a good thing). One important motivation for Nickel is all these new use-cases for configuration that just didn't exist before, and they bring in complexity. There, JSON and co becomes verbose, hard to maintain, hard to "debug", and so on.

The first symptom is you ending up duplicating a lot of data that may become out of sync (like IPs, machine names, servers, etc.). Just by adding variables (let-binding) and say string interpolation, you already gain a lot by being able to specifying a single source of truth and refers to it elsewhere. With functions, you are able to describe dependencies between data: typically, the list of open ports is derived from the list of enabled protocols, e.g.

@ysndr ysndr Jan 4, 2022

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's covered in parts in another subsection

Comment thread chapter/background.md Outdated
Comment on lines +142 to +99
Despite this, not all languages serve as a configuration language, e.g. compiled languages and some domains require language agnostic formats.
For particularly complex products, both language independence and advanced features are desirable.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can mention it. Most of the config languages I know and you cite are pure, that is they can't perform side effects like interacting with the system. It's an important property for both reproducibility (avoid having the configuration unseemly depends on external state) and security (you don't want your config to be able to run arbitrary programs).

Comment thread chapter/background.md Outdated
Comment thread chapter/background.md Outdated
Alternatively to generating configurations using high level languages, this demand is addressed by more domain specific languages.
Dhall [@dhall], Cue [@cue] or jsonnet [@jsonnet] are such intermediate languages, that offer varying support for string interpolation, (strict) typing, functions and validation.

## Infrastructure as Code

@yannham yannham Jan 3, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you intend to have a specific section dedicated to this. Then I would put this section before the previous one, as it serves as context about why configuration languages are needed. IaC is one of the main motivation and use-case.

Comment thread chapter/background.md
For particularly complex products, both language independence and advanced features are desirable.
Alternatively to generating configurations using high level languages, this demand is addressed by more domain specific languages.
Dhall [@dhall], Cue [@cue] or jsonnet [@jsonnet] are such intermediate languages, that offer varying support for string interpolation, (strict) typing, functions and validation.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should probably also cite Nix, as it is one of the main target of Nickel. Don't have to explain everything, but mention it is a package manager enforcing reproducibility that uses a language to describe its software packages.

@github-actions

github-actions Bot commented Jan 4, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

3 similar comments
@github-actions

github-actions Bot commented Jan 4, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

github-actions Bot commented Jan 4, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@github-actions

github-actions Bot commented Jan 4, 2022

Copy link
Copy Markdown

Uploaded preview of this chapter and the full document preview branch

Preview: chapter/background

@ysndr
ysndr requested a review from yannham January 5, 2022 16:03
@ysndr

ysndr commented Jan 5, 2022

Copy link
Copy Markdown
Owner Author

@yannham I addressed most of your remarks, if you want to have an eye on it
Im working on the methods now and find myself writing a couple of nickel internals in that chapter about which I think they would be to technical in the background (elements of the ast t.ex)

@ysndr ysndr mentioned this pull request Jan 11, 2022
3 tasks
ysndr and others added 30 commits April 9, 2022 21:13
Co-authored-by: Martin Monperrus <martin.monperrus@gnieh.org>
Co-authored-by: Martin Monperrus <martin.monperrus@gnieh.org>
Co-authored-by: Martin Monperrus <martin.monperrus@gnieh.org>
Co-authored-by: Martin Monperrus <martin.monperrus@gnieh.org>
(cherry picked from commit 4c3c83c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-preview Pause preview generation for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants