Module input

Input capability for nom combinators to parse

Input types include:

• &str and &[u8] are the standard input types
• Located can track the location within the original buffer to report spans
• Stateful to thread global state through your parsers
• Streaming can mark an input as partial buffer that is being streamed into

How do a parse a custom input type?

While historically, nom has worked mainly on &[u8] and &str, it can actually use any type as input, as long as they follow a specific set of traits. Those traits were developed first to abstract away the differences between &[u8] and &str, but were then employed for more interesting types, like nom_locate, a wrapper type that can carry line and column information, or to parse a list of tokens.

Implementing a custom type

Let’s assume we have an input type we’ll call MyInput. MyInput is a sequence of MyItem type. The goal is to define nom parsers with this signature: MyInput -> IResult<MyInput, Output>.

fn parser(i: MyInput) -> IResult<MyInput, Output> {
    tag("test")(i)
}

Here are the traits we have to implement for MyInput:

trait	usage
InputIsStreaming	Marks the input as being the complete buffer or a partial buffer for streaming input
AsBytes	Casts the input type to a byte slice
Compare	Character comparison operations
ExtendInto	Abstracts something which can extend an Extend
FindSubstring	Look for a substring in self
FindToken	Look for self in the given input stream
InputIter	Common iteration operations on the input type
InputLength	Calculate the input length
IntoOutput	Adapt a captired Input into an appropriate type
Location	Calculate location within initial input
InputTake	Slicing operations
InputTakeAtPosition	Look for a specific token and split at its position
Offset	Calculate the offset between slices
ParseTo	Used to integrate &str’s parse() method
Slice	Slicing operations using ranges

Here are the traits we have to implement for MyItem:

trait	usage
AsChar	Transforms common types to a char for basic token parsing

Structs

Located

Allow collecting the span of a parsed token

Stateful

Thread global state through your parsers

Streaming

Mark the input as a partial buffer for streaming input.

Enums

CompareResult

Indicates whether a comparison was successful, an error, or if more data was needed

Traits

AsBytes

Helper trait for types that can be viewed as a byte slice

AsChar

Transforms common types to a char for basic token parsing

Compare

Abstracts comparison operations

ErrorConvert

Equivalent From implementation to avoid orphan rules in bits parsers

ExtendInto

Abstracts something which can extend an Extend. Used to build modified input slices in escaped_transform

FindSubstring

Look for a substring in self

FindToken

Check if a token in in a set of possible tokens

HexDisplay

Helper trait to show a byte slice as a hex dump

InputIsStreaming

Marks the input as being the complete buffer or a partial buffer for streaming input

InputIter

Abstracts common iteration operations on the input type

InputLength

Abstract method to calculate the input length

InputTake

Abstracts slicing operations

InputTakeAtPosition

Methods to take as much input as possible until the provided function returns true for the current element.

IntoOutput

Convert an Input into an appropriate Output type

Location

Number of indices input has advanced since start of parsing

Offset

Useful functions to calculate the offset between slices and show a hexdump of a slice

ParseTo

Used to integrate str’s parse() method

Slice

Slicing operations using ranges.

ToUsize

Helper trait to convert numbers to usize.

UnspecializedInput

Dummy trait used for default implementations (currently only used for InputTakeAtPosition and Compare).