Switch parser to multi-byte processing

This patch overhauls the `Parser::advance` API to operate on byte slices
instead of individual bytes, which allows for additional performance
optimizations.

VTE does not support C1 escapes and C0 escapes always start with an
escape character. This makes it possible to simplify processing if a
byte stream is determined to not contain any escapes. The `memchr` crate
provides a battle-tested implementation for SIMD-accelerated byte
searches, which is why this implementation makes use of it.

VTE also only supports UTF8 characters in the ground state, which means
that the new non-escape parsing path is able to rely completely on STD's
`str::from_utf8` since `memchr` gives us the full length of the plain
text character buffer. This allows us to completely remove `utf8parse`
and all related code.

We also make use of `memchr` in the synchronized escape handling in
`ansi.rs`, since it relies heavily on scanning large amounts of text for
the extension/termination escape sequences.

2 files changed, 0 insertions, 243 deletions

diff --git a/utf8parse/tests/UTF-8-demo.txt b/utf8parse/tests/UTF-8-demo.txt
deleted file mode 100644
index 4363f27..0000000
--- a/utf8parse/tests/UTF-8-demo.txt
+++ /dev/null
@@ -1,212 +0,0 @@
-
-UTF-8 encoded sample plain-text file
-‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾
-
-Markus Kuhn [ˈmaʳkʊs kuːn] <http://www.cl.cam.ac.uk/~mgk25/> — 2002-07-25
-
-
-The ASCII compatible UTF-8 encoding used in this plain-text file
-is defined in Unicode, ISO 10646-1, and RFC 2279.
-
-
-Using Unicode/UTF-8, you can write in emails and source code things such as
-
-Mathematics and sciences:
-
-  ∮ E⋅da = Q,  n → ∞, ∑ f(i) = ∏ g(i),      ⎧⎡⎛┌─────┐⎞⎤⎫
-                                            ⎪⎢⎜│a²+b³ ⎟⎥⎪
-  ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β),    ⎪⎢⎜│───── ⎟⎥⎪
-                                            ⎪⎢⎜⎷ c₈   ⎟⎥⎪
-  ℕ ⊆ ℕ₀ ⊂ ℤ ⊂ ℚ ⊂ ℝ ⊂ ℂ,                   ⎨⎢⎜       ⎟⎥⎬
-                                            ⎪⎢⎜ ∞     ⎟⎥⎪
-  ⊥ < a ≠ b ≡ c ≤ d ≪ ⊤ ⇒ (⟦A⟧ ⇔ ⟪B⟫),      ⎪⎢⎜ ⎲     ⎟⎥⎪
-                                            ⎪⎢⎜ ⎳aⁱ-bⁱ⎟⎥⎪
-  2H₂ + O₂ ⇌ 2H₂O, R = 4.7 kΩ, ⌀ 200 mm     ⎩⎣⎝i=1    ⎠⎦⎭
-
-Linguistics and dictionaries:
-
-  ði ıntəˈnæʃənəl fəˈnɛtık əsoʊsiˈeıʃn
-  Y [ˈʏpsilɔn], Yen [jɛn], Yoga [ˈjoːgɑ]
-
-APL:
-
-  ((V⍳V)=⍳⍴V)/V←,V    ⌷←⍳→⍴∆∇⊃‾⍎⍕⌈
-
-Nicer typography in plain text files:
-
-  ╔══════════════════════════════════════════╗
-  ║                                          ║
-  ║   • ‘single’ and “double” quotes         ║
-  ║                                          ║
-  ║   • Curly apostrophes: “We’ve been here” ║
-  ║                                          ║
-  ║   • Latin-1 apostrophe and accents: '´`  ║
-  ║                                          ║
-  ║   • ‚deutsche‘ „Anführungszeichen“       ║
-  ║                                          ║
-  ║   • †, ‡, ‰, •, 3–4, —, −5/+5, ™, …      ║
-  ║                                          ║
-  ║   • ASCII safety test: 1lI|, 0OD, 8B     ║
-  ║                      ╭─────────╮         ║
-  ║   • the euro symbol: │ 14.95 € │         ║
-  ║                      ╰─────────╯         ║
-  ╚══════════════════════════════════════════╝
-
-Combining characters:
-
-  STARGΛ̊TE SG-1, a = v̇ = r̈, a⃑ ⊥ b⃑
-
-Greek (in Polytonic):
-
-  The Greek anthem:
-
-  Σὲ γνωρίζω ἀπὸ τὴν κόψη
-  τοῦ σπαθιοῦ τὴν τρομερή,
-  σὲ γνωρίζω ἀπὸ τὴν ὄψη
-  ποὺ μὲ βία μετράει τὴ γῆ.
-
-  ᾿Απ᾿ τὰ κόκκαλα βγαλμένη
-  τῶν ῾Ελλήνων τὰ ἱερά
-  καὶ σὰν πρῶτα ἀνδρειωμένη
-  χαῖρε, ὦ χαῖρε, ᾿Ελευθεριά!
-
-  From a speech of Demosthenes in the 4th century BC:
-
-  Οὐχὶ ταὐτὰ παρίσταταί μοι γιγνώσκειν, ὦ ἄνδρες ᾿Αθηναῖοι,
-  ὅταν τ᾿ εἰς τὰ πράγματα ἀποβλέψω καὶ ὅταν πρὸς τοὺς
-  λόγους οὓς ἀκούω· τοὺς μὲν γὰρ λόγους περὶ τοῦ
-  τιμωρήσασθαι Φίλιππον ὁρῶ γιγνομένους, τὰ δὲ πράγματ᾿
-  εἰς τοῦτο προήκοντα,  ὥσθ᾿ ὅπως μὴ πεισόμεθ᾿ αὐτοὶ
-  πρότερον κακῶς σκέψασθαι δέον. οὐδέν οὖν ἄλλο μοι δοκοῦσιν
-  οἱ τὰ τοιαῦτα λέγοντες ἢ τὴν ὑπόθεσιν, περὶ ἧς βουλεύεσθαι,
-  οὐχὶ τὴν οὖσαν παριστάντες ὑμῖν ἁμαρτάνειν. ἐγὼ δέ, ὅτι μέν
-  ποτ᾿ ἐξῆν τῇ πόλει καὶ τὰ αὑτῆς ἔχειν ἀσφαλῶς καὶ Φίλιππον
-  τιμωρήσασθαι, καὶ μάλ᾿ ἀκριβῶς οἶδα· ἐπ᾿ ἐμοῦ γάρ, οὐ πάλαι
-  γέγονεν ταῦτ᾿ ἀμφότερα· νῦν μέντοι πέπεισμαι τοῦθ᾿ ἱκανὸν
-  προλαβεῖν ἡμῖν εἶναι τὴν πρώτην, ὅπως τοὺς συμμάχους
-  σώσομεν. ἐὰν γὰρ τοῦτο βεβαίως ὑπάρξῃ, τότε καὶ περὶ τοῦ
-  τίνα τιμωρήσεταί τις καὶ ὃν τρόπον ἐξέσται σκοπεῖν· πρὶν δὲ
-  τὴν ἀρχὴν ὀρθῶς ὑποθέσθαι, μάταιον ἡγοῦμαι περὶ τῆς
-  τελευτῆς ὁντινοῦν ποιεῖσθαι λόγον.
-
-  Δημοσθένους, Γ´ ᾿Ολυνθιακὸς
-
-Georgian:
-
-  From a Unicode conference invitation:
-
-  გთხოვთ ახლავე გაიაროთ რეგისტრაცია Unicode-ის მეათე საერთაშორისო
-  კონფერენციაზე დასასწრებად, რომელიც გაიმართება 10-12 მარტს,
-  ქ. მაინცში, გერმანიაში. კონფერენცია შეჰკრებს ერთად მსოფლიოს
-  ექსპერტებს ისეთ დარგებში როგორიცაა ინტერნეტი და Unicode-ი,
-  ინტერნაციონალიზაცია და ლოკალიზაცია, Unicode-ის გამოყენება
-  ოპერაციულ სისტემებსა, და გამოყენებით პროგრამებში, შრიფტებში,
-  ტექსტების დამუშავებასა და მრავალენოვან კომპიუტერულ სისტემებში.
-
-Russian:
-
-  From a Unicode conference invitation:
-
-  Зарегистрируйтесь сейчас на Десятую Международную Конференцию по
-  Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии.
-  Конференция соберет широкий круг экспертов по  вопросам глобального
-  Интернета и Unicode, локализации и интернационализации, воплощению и
-  применению Unicode в различных операционных системах и программных
-  приложениях, шрифтах, верстке и многоязычных компьютерных системах.
-
-Thai (UCS Level 2):
-
-  Excerpt from a poetry on The Romance of The Three Kingdoms (a Chinese
-  classic 'San Gua'):
-
-  [----------------------------|------------------------]
-    ๏ แผ่นดินฮั่นเสื่อมโทรมแสนสังเวช  พระปกเกศกองบู๊กู้ขึ้นใหม่
-  สิบสองกษัตริย์ก่อนหน้าแลถัดไป       สององค์ไซร้โง่เขลาเบาปัญญา
-    ทรงนับถือขันทีเป็นที่พึ่ง           บ้านเมืองจึงวิปริตเป็นนักหนา
-  โฮจิ๋นเรียกทัพทั่วหัวเมืองมา         หมายจะฆ่ามดชั่วตัวสำคัญ
-    เหมือนขับไสไล่เสือจากเคหา      รับหมาป่าเข้ามาเลยอาสัญ
-  ฝ่ายอ้องอุ้นยุแยกให้แตกกัน          ใช้สาวนั้นเป็นชนวนชื่นชวนใจ
-    พลันลิฉุยกุยกีกลับก่อเหตุ          ช่างอาเพศจริงหนาฟ้าร้องไห้
-  ต้องรบราฆ่าฟันจนบรรลัย           ฤๅหาใครค้ำชูกู้บรรลังก์ ฯ
-
-  (The above is a two-column text. If combining characters are handled
-  correctly, the lines of the second column should be aligned with the
-  | character above.)
-
-Ethiopian:
-
-  Proverbs in the Amharic language:
-
-  ሰማይ አይታረስ ንጉሥ አይከሰስ።
-  ብላ ካለኝ እንደአባቴ በቆመጠኝ።
-  ጌጥ ያለቤቱ ቁምጥና ነው።
-  ደሀ በሕልሙ ቅቤ ባይጠጣ ንጣት በገደለው።
-  የአፍ ወለምታ በቅቤ አይታሽም።
-  አይጥ በበላ ዳዋ ተመታ።
-  ሲተረጉሙ ይደረግሙ።
-  ቀስ በቀስ፥ ዕንቁላል በእግሩ ይሄዳል።
-  ድር ቢያብር አንበሳ ያስር።
-  ሰው እንደቤቱ እንጅ እንደ ጉረቤቱ አይተዳደርም።
-  እግዜር የከፈተውን ጉሮሮ ሳይዘጋው አይድርም።
-  የጎረቤት ሌባ፥ ቢያዩት ይስቅ ባያዩት ያጠልቅ።
-  ሥራ ከመፍታት ልጄን ላፋታት።
-  ዓባይ ማደሪያ የለው፥ ግንድ ይዞ ይዞራል።
-  የእስላም አገሩ መካ የአሞራ አገሩ ዋርካ።
-  ተንጋሎ ቢተፉ ተመልሶ ባፉ።
-  ወዳጅህ ማር ቢሆን ጨርስህ አትላሰው።
-  እግርህን በፍራሽህ ልክ ዘርጋ።
-
-Runes:
-
-  ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ
-
-  (Old English, which transcribed into Latin reads 'He cwaeth that he
-  bude thaem lande northweardum with tha Westsae.' and means 'He said
-  that he lived in the northern land near the Western Sea.')
-
-Braille:
-
-  ⡌⠁⠧⠑ ⠼⠁⠒  ⡍⠜⠇⠑⠹⠰⠎ ⡣⠕⠌
-
-  ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠙⠑⠁⠙⠒ ⠞⠕ ⠃⠑⠛⠔ ⠺⠊⠹⠲ ⡹⠻⠑ ⠊⠎ ⠝⠕ ⠙⠳⠃⠞
-  ⠱⠁⠞⠑⠧⠻ ⠁⠃⠳⠞ ⠹⠁⠞⠲ ⡹⠑ ⠗⠑⠛⠊⠌⠻ ⠕⠋ ⠙⠊⠎ ⠃⠥⠗⠊⠁⠇ ⠺⠁⠎
-  ⠎⠊⠛⠝⠫ ⠃⠹ ⠹⠑ ⠊⠇⠻⠛⠹⠍⠁⠝⠂ ⠹⠑ ⠊⠇⠻⠅⠂ ⠹⠑ ⠥⠝⠙⠻⠞⠁⠅⠻⠂
-  ⠁⠝⠙ ⠹⠑ ⠡⠊⠑⠋ ⠍⠳⠗⠝⠻⠲ ⡎⠊⠗⠕⠕⠛⠑ ⠎⠊⠛⠝⠫ ⠊⠞⠲ ⡁⠝⠙
-  ⡎⠊⠗⠕⠕⠛⠑⠰⠎ ⠝⠁⠍⠑ ⠺⠁⠎ ⠛⠕⠕⠙ ⠥⠏⠕⠝ ⠰⡡⠁⠝⠛⠑⠂ ⠋⠕⠗ ⠁⠝⠹⠹⠔⠛ ⠙⠑
-  ⠡⠕⠎⠑ ⠞⠕ ⠏⠥⠞ ⠙⠊⠎ ⠙⠁⠝⠙ ⠞⠕⠲
-
-  ⡕⠇⠙ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲
-
-  ⡍⠔⠙⠖ ⡊ ⠙⠕⠝⠰⠞ ⠍⠑⠁⠝ ⠞⠕ ⠎⠁⠹ ⠹⠁⠞ ⡊ ⠅⠝⠪⠂ ⠕⠋ ⠍⠹
-  ⠪⠝ ⠅⠝⠪⠇⠫⠛⠑⠂ ⠱⠁⠞ ⠹⠻⠑ ⠊⠎ ⠏⠜⠞⠊⠊⠥⠇⠜⠇⠹ ⠙⠑⠁⠙ ⠁⠃⠳⠞
-  ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡊ ⠍⠊⠣⠞ ⠙⠁⠧⠑ ⠃⠑⠲ ⠔⠊⠇⠔⠫⠂ ⠍⠹⠎⠑⠇⠋⠂ ⠞⠕
-  ⠗⠑⠛⠜⠙ ⠁ ⠊⠕⠋⠋⠔⠤⠝⠁⠊⠇ ⠁⠎ ⠹⠑ ⠙⠑⠁⠙⠑⠌ ⠏⠊⠑⠊⠑ ⠕⠋ ⠊⠗⠕⠝⠍⠕⠝⠛⠻⠹
-  ⠔ ⠹⠑ ⠞⠗⠁⠙⠑⠲ ⡃⠥⠞ ⠹⠑ ⠺⠊⠎⠙⠕⠍ ⠕⠋ ⠳⠗ ⠁⠝⠊⠑⠌⠕⠗⠎
-  ⠊⠎ ⠔ ⠹⠑ ⠎⠊⠍⠊⠇⠑⠆ ⠁⠝⠙ ⠍⠹ ⠥⠝⠙⠁⠇⠇⠪⠫ ⠙⠁⠝⠙⠎
-  ⠩⠁⠇⠇ ⠝⠕⠞ ⠙⠊⠌⠥⠗⠃ ⠊⠞⠂ ⠕⠗ ⠹⠑ ⡊⠳⠝⠞⠗⠹⠰⠎ ⠙⠕⠝⠑ ⠋⠕⠗⠲ ⡹⠳
-  ⠺⠊⠇⠇ ⠹⠻⠑⠋⠕⠗⠑ ⠏⠻⠍⠊⠞ ⠍⠑ ⠞⠕ ⠗⠑⠏⠑⠁⠞⠂ ⠑⠍⠏⠙⠁⠞⠊⠊⠁⠇⠇⠹⠂ ⠹⠁⠞
-  ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲
-
-  (The first couple of paragraphs of "A Christmas Carol" by Dickens)
-
-Compact font selection example text:
-
-  ABCDEFGHIJKLMNOPQRSTUVWXYZ /0123456789
-  abcdefghijklmnopqrstuvwxyz £©µÀÆÖÞßéöÿ
-  –—‘“”„†•…‰™œŠŸž€ ΑΒΓΔΩαβγδω АБВГДабвгд
-  ∀∂∈ℝ∧∪≡∞ ↑↗↨↻⇣ ┐┼╔╘░►☺♀ fi�⑀₂ἠḂӥẄɐː⍎אԱა
-
-Greetings in various languages:
-
-  Hello world, Καλημέρα κόσμε, コンニチハ
-
-Box drawing alignment tests:                                          █
-                                                                      ▉
-  ╔══╦══╗  ┌──┬──┐  ╭──┬──╮  ╭──┬──╮  ┏━━┳━━┓  ┎┒┏┑   ╷  ╻ ┏┯┓ ┌┰┐    ▊ ╱╲╱╲╳╳╳
-  ║┌─╨─┐║  │╔═╧═╗│  │╒═╪═╕│  │╓─╁─╖│  ┃┌─╂─┐┃  ┗╃╄┙  ╶┼╴╺╋╸┠┼┨ ┝╋┥    ▋ ╲╱╲╱╳╳╳
-  ║│╲ ╱│║  │║   ║│  ││ │ ││  │║ ┃ ║│  ┃│ ╿ │┃  ┍╅╆┓   ╵  ╹ ┗┷┛ └┸┘    ▌ ╱╲╱╲╳╳╳
-  ╠╡ ╳ ╞╣  ├╢   ╟┤  ├┼─┼─┼┤  ├╫─╂─╫┤  ┣┿╾┼╼┿┫  ┕┛┖┚     ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▍ ╲╱╲╱╳╳╳
-  ║│╱ ╲│║  │║   ║│  ││ │ ││  │║ ┃ ║│  ┃│ ╽ │┃  ░░▒▒▓▓██ ┊  ┆ ╎ ╏  ┇ ┋ ▎
-  ║└─╥─┘║  │╚═╤═╝│  │╘═╪═╛│  │╙─╀─╜│  ┃└─╂─┘┃  ░░▒▒▓▓██ ┊  ┆ ╎ ╏  ┇ ┋ ▏
-  ╚══╩══╝  └──┴──┘  ╰──┴──╯  ╰──┴──╯  ┗━━┻━━┛  ▗▄▖▛▀▜   └╌╌┘ ╎ ┗╍╍┛ ┋  ▁▂▃▄▅▆▇█
-                                               ▝▀▘▙▄▟
diff --git a/utf8parse/tests/utf-8-demo.rs b/utf8parse/tests/utf-8-demo.rs
deleted file mode 100644
index 51df492..0000000
--- a/utf8parse/tests/utf-8-demo.rs
+++ /dev/null
@@ -1,31 +0,0 @@
-use utf8parse::{Parser, Receiver};
-
-static UTF8_DEMO: &[u8] = include_bytes!("UTF-8-demo.txt");
-
-#[derive(Debug, PartialEq)]
-struct StringWrapper(String);
-
-impl Receiver for StringWrapper {
-    fn codepoint(&mut self, c: char) {
-        self.0.push(c);
-    }
-
-    fn invalid_sequence(&mut self) {}
-}
-
-#[test]
-fn utf8parse_test() {
-    let mut parser = Parser::new();
-
-    // utf8parse implementation
-    let mut actual = StringWrapper(String::new());
-
-    for byte in UTF8_DEMO {
-        parser.advance(&mut actual, *byte)
-    }
-
-    // standard library implementation
-    let expected = String::from_utf8_lossy(UTF8_DEMO);
-
-    assert_eq!(actual.0, expected);
-}