✨ Add basic library functionality

[LesterEvSe]

Added

• Support for parsing use and pub keywords.
• The --dep flag to load external libraries.
• The driver directory to resolve external libraries.
• The C3 Linearization algorithm to resolve the library order within the driver directory.
• Test coverage for the new functionality.

Updated

Change the ast.rs file in accordance with the changes mention above.

Note: This PR contains a basic implementation of library support. Therefore, improved error display, loading multiple libraries, and handling reserved words will be addressed in future PR

[KyrylR]

Suggested change

	#[derive(Clone, Debug, PartialEq, Eq, Hash)]
	#[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
	pub enum Visibility {
	Public,
	Private,
	}
	#[derive(Clone, Debug, PartialEq, Eq, Hash, Default)]
	#[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
	pub enum Visibility {
	Public,
	#[default]
	Private,
	}

[delta1]

@KyrylR please re-review

[apoelstra]

99bff73 needs rebase

[KyrylR]

@KyrylR please re-review

Sure, on it, though it will take time, as its quite a big feature

[stringhandler]

3dd5f03

I'm sorry I've taken so long to review this. I do like the concept and the driver.
I would like to suggest these changes:

1. I suggest we distinguish between local modules and package manager includes. There is no package manager at the moment, but there may be one in future.
   For local files I suggest requiring the mod keyword in the entry point file for files that need to be included. Then I suggest we introduce a crate-like keyword for referencing items that are local to the project (in the same package).

// main.simf
mod math;

// reference the math mod inside this package
use crate::math::simple_op::hash;

// versus
// reference the math package from a package manager
use math::simple_op::hash;

2. I suggest we pre-emptively choose a name for simplicityhl packages and create a Cargo.toml like package description now. Working with the symphony idea, perhaps I can suggest part, motif,movement for the crate replacement, and composer,orchestrate, arrange for the Cargo replacement.
   This is where we ask the question: Should this happen here or in simplex, which also has a simplex.toml? Rustc and Cargo have different roles, but I would argue that the case here is different. To my knowledge, rustc compiles individual files or mods and then they are linked. We are not linking after compile, but rather before analysis in the AST. So I would suggest that the package definition (cargo.toml) should be the responsibility of simc and not simplex.

That said, perhaps smplx can have the role of package manager in future and allow downloading packages (crates) from repositories.

[stringhandler]

I am not a fan of this approach. The registry can be cleaned at any time.

Do we even need to suggest a way of doing this?

[stringhandler]

if we are going to suggest, I would suggest using git clone or http download

[LesterEvSe]

Okay, I can either add an additional approach using Git Clone and emphasise that it is basic, or I can simply replace it entirely. I prefer the first approach. What do you think?

This is a question for @KyrylR, too.

[stringhandler]

I would say, don't recommend any method for downloading, let the user decide.

[stringhandler]

This also seems more complicated than using git or http

[LesterEvSe]

Same as above

[LesterEvSe]

2. I suggest we pre-emptively choose a name for `simplicityhl` packages and create a `Cargo.toml` like package description now. Working with the symphony idea, perhaps I can suggest `part`, `motif`,`movement` for the `crate` replacement, and `composer`,`orchestrate`, `arrange` for the `Cargo` replacement.
   This is where we ask the question: Should this happen here or in [simplex](https://github.com/BlockstreamResearch/smplx), which also has a `simplex.toml`? Rustc and Cargo have different roles, but I would argue that the case here is different. To my knowledge, `rustc` compiles individual files or mods and then they are linked. We are not linking after compile, but rather before analysis in the AST. So I would suggest that the package definition (cargo.toml) should be the responsibility of `simc` and not `simplex`.

That said, perhaps smplx can have the role of package manager in future and allow downloading packages (crates) from repositories.

I will answer the second question right away. Based on this Simplex issue #38, we assume that the load functionality will be in Simplex.

[LesterEvSe]

1. I suggest we distinguish between local modules and package manager includes. There is no package manager at the moment, but there may be one in future.
   For local files I suggest requiring the `mod` keyword in the entry point file for files that need to be included. Then I suggest we introduce a `crate`-like keyword for referencing items that are local to the project (in the same package).

// main.simf
mod math;

// reference the math mod inside this package
use crate::math::simple_op::hash;

// versus
// reference the math package from a package manager
use math::simple_op::hash;

Thanks for your review! :)
Regarding this idea, I think it is good and could be implemented in a future PR. However, for this PR it would be excessive, because we are not implementing the mod keyword at all yet, and it would require additional changes to the resolve strategy. In my opinion, it would be better to finalize and merge the current functionality with imports, and then start thinking about mod, because it has a large scope.

Could you open an issue and detail your proposed approach for the mod keyword so we can track it?

[stringhandler]

Thanks, I will open an issue. I would say that this should be resolved before we merge this though, because otherwise there will be conflicting meanings in future.

For example, if we merge this pr as is, then use std::hashing; could be valid in files, and later on will not be valid.

In order to not incur massive changes to this PR, maybe we can settle on always requiring crate in front of the use.

use crate::math::ops;

I personally don't like crate and would prefer one of the above names, but I prefer that over having the non-crate syntax being merged in main for a bit.

[LesterEvSe]

Thanks, I will open an issue. I would say that this should be resolved before we merge this though, because otherwise there will be conflicting meanings in future.

For example, if we merge this pr as is, then use std::hashing; could be valid in files, and later on will not be valid.

In order to not incur massive changes to this PR, maybe we can settle on always requiring crate in front of the use.

use crate::math::ops;

I personally don't like crate and would prefer one of the above names, but I prefer that over having the non-crate syntax being merged in main for a bit.

Hmm, okay, that makes sense. I will think about the best way to implement it