PEG Parser Library for Pony

I have recently released a PEG parser library for the Pony programming language: Kiuatan (“horse” or “pony” in Chinook Jargon). This is mostly a learning exercise for me. It doesn’t make use of any of Pony’s actor model functionality.

Kiuatan (“horse” or “pony” in Chinook Jargon) is a library for building and running parsers in the Pony programming language.

Kiuatan uses Parsing Expression Grammar semantics, which means:

  • Choices are ordered, i.e. the parser will always try to parse alternatives in the order they are declared.
    • Parsers do not backtrack from successful choices.
    • Sequences are greedy, i.e. the parser will not backtrack from the end of a sequence.
  • Kiuatan parsers are “packrat” parsers; they memoize intermediate results, resulting in linear-time parsing.
  • Parsers use Mederios et al’s algorithm to handle unlimited left-recursion.

IronMeta 4.3.0 released

I have released a new version (4.3.0) of IronMeta. Now you can pass arbitrary patterns as arguments to rules. Variables will be captured.

The IronMeta parser generator provides a programming language and application for generating pattern matchers on arbitrary streams of objects. It is an implementation of Alessandro Warth’s OMeta system in C#.

Building the Pony compiler on Windows

Update 2017-01

I have made a vastly simpler build system for Pony on Windows.

Original

I have been taking a look at the very interesting Pony programming language in my copious free time for a couple of weeks now. It’s fairly straightforward to build on Ubuntu. Here’s how I built it on Windows 10:

This process uses GnuWin32, which LLVM recommends, to provide unix utils; your mileage may vary with MSYS or Cygwin.

Build LLVM:

  • Install GnuWin32 via the GetGnuWin32 tool; make sure it’s in your PATH.
  • Install Python (I used 3.5.1; 2.7 is supposed to work as well); make sure it’s in PATH.
  • Installed CMake; make sure it’s in PATH.
  • Get the LLVM source. I used the tarball of 3.7.1 (requireds 7-zip to open on Windows).
  • Make sure you have VS2015 with C++ installed.
  • Generate LLVM VS2015 configuration with CMake. I used the GUI to configure and generate the VS projects; make sure you use the 64-bit generator (“Visual Studio 14 2015 Win64”), and set the CMAKE_INSTALL_PREFIX to where you want LLVM to live.
  • Build the INSTALL project in the LLVM solution.

Build PonyC:

  • Get the PreMake5 executable, and put it somewhere in your PATH.
  • Get the PonyC source.
  • Run premake5.exe --with-tests --to=..\vs vs2015 to generate Ponyc.sln.
  • Changed the Character Set property of each project in the PonyC solution to Not Set.
  • Build Pony.sln

In order to run the pony compiler, you’ll need a few libraries in your environment. I have made a utility that will get the libraries and set up your environment:

  • Install 7-Zip, make sure it’s in PATH.
  • Open a VS2015 x64 Native Tools Command Prompt (things will not work correctly otherwise!) and run:


    > git clone git@github.com:kulibali/ponyc-windows-libs.git
    > cd ponyc-windows-libs
    > .\getlibs.bat
    > .\setenv.bat

  • Now you can run the pony compiler and tests:


    > cd path_to_pony_source
    > build\release\testc.exe
    > build\release\testrt.exe
    > build\release\ponyc.exe -d -s packages\stdlib
    > .\stdlib

System.Security.Claims .ClaimsIdentity.WriteTo() has a bug

Getting “cannot read past the end of the stream” when trying to use System.Security.ClaimsIdentity.WriteTo() to store a claims identity. Disassembled it:

// System.Security.Claims.ClaimsIdentity
protected virtual void WriteTo(BinaryWriter writer, byte[] userData)
{
	if (writer == null)
	{
		throw new ArgumentNullException("writer");
	}
	int num = 0;
	ClaimsIdentity.SerializationMask serializationMask = ClaimsIdentity.SerializationMask.None;
        ... get flags here...
	writer.Write((int)serializationMask);
	writer.Write(num);
	if ((serializationMask & ClaimsIdentity.SerializationMask.AuthenticationType) == ClaimsIdentity.SerializationMask.AuthenticationType)
	{
		writer.Write(this.m_authenticationType);
	}
        ...
	if ((serializationMask & ClaimsIdentity.SerializationMask.HasClaims) == ClaimsIdentity.SerializationMask.HasClaims)
	{
		writer.Write(this.m_instanceClaims.Count);
		using (List<Claim>.Enumerator enumerator = this.m_instanceClaims.GetEnumerator())
		{
			while (enumerator.MoveNext())
			{
				enumerator.Current.WriteTo(writer);
			}
		}
	}
        ...
	writer.Flush();
}

It writes the flags, then the number of fields, then some of the other values, then the number of claims, then the claims, then some more fields.

On the other hand, the constructor doesn’t read the number of fields, just starts reading the fields (but not Label) and then the claims, and then doesn’t read the rest of the fields.

// System.Security.Claims.ClaimsIdentity
private void Initialize(BinaryReader reader)
{
	if (reader == null)
	{
		throw new ArgumentNullException("reader");
	}
	int expr_14 = reader.ReadInt32();
	if ((expr_14 & 1) == 1)
	{
		this.m_authenticationType = reader.ReadString();
	}
	if ((expr_14 & 2) == 2)
	{
		this.m_bootstrapContext = reader.ReadString();
	}
	if ((expr_14 & 4) == 4)
	{
		this.m_nameType = reader.ReadString();
	}
	else
	{
		this.m_nameType = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name";
	}
	if ((expr_14 & 8) == 8)
	{
		this.m_roleType = reader.ReadString();
	}
	else
	{
		this.m_roleType = "http://schemas.microsoft.com/ws/2008/06/identity/claims/role";
	}
	if ((expr_14 & 16) == 16)
	{
		int num = reader.ReadInt32();
		for (int i = 0; i < num; i++)
		{
			Claim item = new Claim(reader, this);
			this.m_instanceClaims.Add(item);
		}
	}
}