More Progress!

I resurrected the test, edoc, clean, and release tasks tonight. Everything works as it should with the exception of edoc. There seems to be a conflict between edoc and xmerl in the newest version of erlang (R11B-3). I hope to figure out what the problem is and resolve it soon.

On a side note. I came across the cover module in the tools application of Erlang. Since I was working on the unit tests task at the time it seemed like a good idea to add this functionality to the build system either via a code coverage task or integration into the existing unit tests task. The only real issue is that all the dependent modules need to be recompiled with special coverage information to make cover work. This adds quite a bit of complexity to the feature. In any case, its on my list of new tasks to add.

I added one task additional, an analyze tasks. It seems the dialyzer folks finally made it controllable from erlang code. This allowed me to integrate dialyzer into sinan. Now running dialyzer will be as simple as running a build task. Hopefully, if dialyzer is easier to use it will get used more often.

Building Again!

I finally finished refactoring the build task tonight. For the first time the latest version of the build system is actually building source. It took a huge amount of effort to get the dependent tasks to this point. The engine, discover, depends, the repo puller, all of it was just preparation for the builder. In any case, I am very pleased.

For the moment it just builds *.erl and *.yrl but eventually I want to support all of the erlang compilables. I suspect that refactoring the rest of the tasks (test, release, tar, clean, etc) wont take much longer. Hopefully, I will be able to do an alpha release at the end of this week. I certainly hope so in any case.

Topological Sort Joy

I finally modified sinan to make use of all the dependency analysis code I wrote for ewrepo. One thing that I needed to do was produce a list, in the dependent order, of the graph of dependencies in the internal project applications. This is essential for compile time things like parse transforms are going to work correctly. I wrestled with how to do this correctly for quite a bit before I realized that its a simple topological sort. I should really have realized this immediately, but I didn't. Fortunately, this isn't a difficult algorithm, especially since I didn't even need to implement it. Joe Armstrong wrote a topo sort for his ermake project and made it available in the contribs section of erlang.org. Once I found that it didn't take to long to integrate it into sinan.

Once I got to thinking about this dependency problem I realized that there was at least one more area that a topological sort was needed. Thats in the run order of the task list. Each task may have any number of tasks that it depends on. Right now my code would just run each dependent task multiple times. Thats a bug, fortunately it wont take more then a few minutes to integrate the topo sort into this area as well.

Dependency Checking Complete!!

I finally finished up both the shallow and the deep dependency resolution for sinan. This took a bit of time and effort to conceptualize, abstract and implement.

Now given a list of apps and the dependencies (along with the url(s) of the repo); like this

  [{app1, "0.1", [{edoc, 'LATEST'},                                                                        
                 {syntax_tools, "1.5.0"}]}

It correctly returns a list of all the dependencies required for the project; like this

 [{stdlib, "1.14.2",
           [compiler,edoc,syntax_tools]},
  {syntax_tools,"1.5.0",
           [app1,edoc]},
  {compiler,"4.4.2",
           [edoc]},
  {kernel,"2.11.2",
           [compiler,stdlib,edoc]},
  {edoc,"0.6.9",
           [app1]},
  {app1,"0.1",[]}]    

If it runs into a version conflict it reports what the conflict is and what applications are causing it.

In the output above. Each entry is a dependency, its version and a list of apps the depend on that app.

Released: Ktuo 0.1.1

I just released the first alpha version of my JSON Parser/Encoder Ktuo. I use it in several projects with no problems at all. However, this is still an alpha release, so be aware that you may run into issues. The download area for Ktuo is here. A bit of documentation for the project is here.

I know the question of why another JSON parser is going to come up, so I am just going to go ahead and address it now. The existing JSON parser is still available and still useful. It makes elegant use of continuations to handle cases where it may not have a full JSON expression available. This makes it very flexible and pretty darn interesting. However, these extra features also add significant complexity and no small amount of additional resource usage. While working on my Tercio project I realized that I didn't need these features and I didn't want to pay the cost of the complexity and resource usage overhead. So I wrote my own. Eventually, I realized it was useful in and of itself and created a project space for it.

Finished Up Dependency Checking

I finished up the shallow dependency checking. Now I just need to integrate with the repository and repository metadata to do deep dependency analysis and package pulling.

More Dependencies

Most people think that Erlang has just runtime dependencies. For
the most part this is true. However, Erlang also has two types of
compile time dependencies. The first is the ".hrl" files that are
included into a module. The second type is ".erl" files that
implement a parse transform. Both of these types of files
represent compile time dependencies. For the most part its simple
enough to make sure the include and code path information is
set. However, its much more difficult to make sure that if one of
these types of files change all of its dependencies are changed
as well. I think that the easiest way to handle this would be to
simply look at the ast of a compiled file during the compilation
process and extract the includes and the parse transform
information from that ast. Unfortunately, it means knowing much
more about a dependent OTP application then the build system
currently knows. It would definitely need to happen after the high
level runtime dependency mapping takes place. I think, for now, I
will stick with just handling the runtime dependencies and making
sure that the build time dependencies have the right path
information available. Once the system gets out in the wild and I
start getting feedback I may modify it to be a little smarter
about the compile time dependencies.