mirror of
https://github.com/edg-l/edlang.git
synced 2024-11-24 00:48:24 +00:00
68 lines
18 KiB
HTML
68 lines
18 KiB
HTML
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="This trait allows the user to customize the heuristic used to determine the relative frequency of a given byte in the dataset being searched."><title>HeuristicFrequencyRank in memchr::arch::all::packedpair - Rust</title><link rel="preload" as="font" type="font/woff2" crossorigin href="../../../../static.files/SourceSerif4-Regular-46f98efaafac5295.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../../../static.files/FiraSans-Regular-018c141bf0843ffd.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../../../static.files/FiraSans-Medium-8f9a781e4970d388.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../../../static.files/SourceCodePro-Regular-562dcc5011b6de7d.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../../../../static.files/SourceCodePro-Semibold-d899c5a5c4aeb14a.ttf.woff2"><link rel="stylesheet" href="../../../../static.files/normalize-76eba96aa4d2e634.css"><link rel="stylesheet" href="../../../../static.files/rustdoc-ac92e1bbe349e143.css"><meta name="rustdoc-vars" data-root-path="../../../../" data-static-root-path="../../../../static.files/" data-current-crate="memchr" data-themes="" data-resource-suffix="" data-rustdoc-version="1.76.0 (07dca489a 2024-02-04)" data-channel="1.76.0" data-search-js="search-2b6ce74ff89ae146.js" data-settings-js="settings-4313503d2e1961c2.js" ><script src="../../../../static.files/storage-f2adc0d6ca4d09fb.js"></script><script defer src="sidebar-items.js"></script><script defer src="../../../../static.files/main-305769736d49e732.js"></script><noscript><link rel="stylesheet" href="../../../../static.files/noscript-feafe1bb7466e4bd.css"></noscript><link rel="alternate icon" type="image/png" href="../../../../static.files/favicon-16x16-8b506e7a72182f1c.png"><link rel="alternate icon" type="image/png" href="../../../../static.files/favicon-32x32-422f7d1d52889060.png"><link rel="icon" type="image/svg+xml" href="../../../../static.files/favicon-2c020d218678b618.svg"></head><body class="rustdoc trait"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><nav class="mobile-topbar"><button class="sidebar-menu-toggle">☰</button></nav><nav class="sidebar"><div class="sidebar-crate"><h2><a href="../../../../memchr/index.html">memchr</a><span class="version">2.7.1</span></h2></div><h2 class="location"><a href="#">HeuristicFrequencyRank</a></h2><div class="sidebar-elems"><section><h3><a href="#required-methods">Required Methods</a></h3><ul class="block"><li><a href="#tymethod.rank">rank</a></li></ul><h3><a href="#foreign-impls">Implementations on Foreign Types</a></h3><ul class="block"><li><a href="#impl-HeuristicFrequencyRank-for-%26R">&'a R</a></li></ul><h3><a href="#implementors">Implementors</a></h3></section><h2><a href="index.html">In memchr::arch::all::packedpair</a></h2></div></nav><div class="sidebar-resizer"></div>
|
||
<main><div class="width-limiter"><nav class="sub"><form class="search-form"><span></span><div id="sidebar-button" tabindex="-1"><a href="../../../../memchr/all.html" title="show sidebar"></a></div><input class="search-input" name="search" aria-label="Run search in the documentation" autocomplete="off" spellcheck="false" placeholder="Click or press ‘S’ to search, ‘?’ for more options…" type="search"><div id="help-button" tabindex="-1"><a href="../../../../help.html" title="help">?</a></div><div id="settings-menu" tabindex="-1"><a href="../../../../settings.html" title="settings"><img width="22" height="22" alt="Change settings" src="../../../../static.files/wheel-7b819b6101059cd0.svg"></a></div></form></nav><section id="main-content" class="content"><div class="main-heading"><h1>Trait <a href="../../../index.html">memchr</a>::<wbr><a href="../../index.html">arch</a>::<wbr><a href="../index.html">all</a>::<wbr><a href="index.html">packedpair</a>::<wbr><a class="trait" href="#">HeuristicFrequencyRank</a><button id="copy-path" title="Copy item path to clipboard"><img src="../../../../static.files/clipboard-7571035ce49a181d.svg" width="19" height="18" alt="Copy item path"></button></h1><span class="out-of-band"><a class="src" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#309-318">source</a> · <button id="toggle-all-docs" title="collapse all docs">[<span>−</span>]</button></span></div><pre class="rust item-decl"><code>pub trait HeuristicFrequencyRank {
|
||
// Required method
|
||
fn <a href="#tymethod.rank" class="fn">rank</a>(&self, byte: <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.u8.html">u8</a>) -> <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.u8.html">u8</a>;
|
||
}</code></pre><details class="toggle top-doc" open><summary class="hideme"><span>Expand description</span></summary><div class="docblock"><p>This trait allows the user to customize the heuristic used to determine the
|
||
relative frequency of a given byte in the dataset being searched.</p>
|
||
<p>The use of this trait can have a dramatic impact on performance depending
|
||
on the type of data being searched. The details of why are explained in the
|
||
docs of <a href="../../../memmem/enum.Prefilter.html" title="enum memchr::memmem::Prefilter"><code>crate::memmem::Prefilter</code></a>. To summarize, the core algorithm uses
|
||
a prefilter to quickly identify candidate matches that are later verified
|
||
more slowly. This prefilter is implemented in terms of trying to find
|
||
<code>rare</code> bytes at specific offsets that will occur less frequently in the
|
||
dataset. While the concept of a <code>rare</code> byte is similar for most datasets,
|
||
there are some specific datasets (like binary executables) that have
|
||
dramatically different byte distributions. For these datasets customizing
|
||
the byte frequency heuristic can have a massive impact on performance, and
|
||
might even need to be done at runtime.</p>
|
||
<p>The default implementation of <code>HeuristicFrequencyRank</code> reads from the
|
||
static frequency table defined in <code>src/memmem/byte_frequencies.rs</code>. This
|
||
is optimal for most inputs, so if you are unsure of the impact of using a
|
||
custom <code>HeuristicFrequencyRank</code> you should probably just use the default.</p>
|
||
<h2 id="example"><a href="#example">Example</a></h2>
|
||
<div class="example-wrap"><pre class="rust rust-example-rendered"><code><span class="kw">use </span>memchr::{
|
||
arch::all::packedpair::HeuristicFrequencyRank,
|
||
memmem::FinderBuilder,
|
||
};
|
||
|
||
<span class="doccomment">/// A byte-frequency table that is good for scanning binary executables.
|
||
</span><span class="kw">struct </span>Binary;
|
||
|
||
<span class="kw">impl </span>HeuristicFrequencyRank <span class="kw">for </span>Binary {
|
||
<span class="kw">fn </span>rank(<span class="kw-2">&</span><span class="self">self</span>, byte: u8) -> u8 {
|
||
<span class="kw">const </span>TABLE: [u8; <span class="number">256</span>] = [
|
||
<span class="number">255</span>, <span class="number">128</span>, <span class="number">61</span>, <span class="number">43</span>, <span class="number">50</span>, <span class="number">41</span>, <span class="number">27</span>, <span class="number">28</span>, <span class="number">57</span>, <span class="number">15</span>, <span class="number">21</span>, <span class="number">13</span>, <span class="number">24</span>, <span class="number">17</span>, <span class="number">17</span>,
|
||
<span class="number">89</span>, <span class="number">58</span>, <span class="number">16</span>, <span class="number">11</span>, <span class="number">7</span>, <span class="number">14</span>, <span class="number">23</span>, <span class="number">7</span>, <span class="number">6</span>, <span class="number">24</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">5</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">7</span>, <span class="number">16</span>,
|
||
<span class="number">68</span>, <span class="number">11</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">88</span>, <span class="number">7</span>, <span class="number">4</span>, <span class="number">4</span>, <span class="number">23</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">8</span>, <span class="number">8</span>, <span class="number">5</span>, <span class="number">10</span>, <span class="number">4</span>, <span class="number">30</span>, <span class="number">11</span>,
|
||
<span class="number">9</span>, <span class="number">24</span>, <span class="number">11</span>, <span class="number">5</span>, <span class="number">5</span>, <span class="number">5</span>, <span class="number">19</span>, <span class="number">11</span>, <span class="number">6</span>, <span class="number">17</span>, <span class="number">9</span>, <span class="number">9</span>, <span class="number">6</span>, <span class="number">8</span>,
|
||
<span class="number">48</span>, <span class="number">58</span>, <span class="number">11</span>, <span class="number">14</span>, <span class="number">53</span>, <span class="number">40</span>, <span class="number">9</span>, <span class="number">9</span>, <span class="number">254</span>, <span class="number">35</span>, <span class="number">3</span>, <span class="number">6</span>, <span class="number">52</span>, <span class="number">23</span>, <span class="number">6</span>, <span class="number">6</span>, <span class="number">27</span>,
|
||
<span class="number">4</span>, <span class="number">7</span>, <span class="number">11</span>, <span class="number">14</span>, <span class="number">13</span>, <span class="number">10</span>, <span class="number">11</span>, <span class="number">11</span>, <span class="number">5</span>, <span class="number">2</span>, <span class="number">10</span>, <span class="number">16</span>, <span class="number">12</span>, <span class="number">6</span>, <span class="number">19</span>,
|
||
<span class="number">19</span>, <span class="number">20</span>, <span class="number">5</span>, <span class="number">14</span>, <span class="number">16</span>, <span class="number">31</span>, <span class="number">19</span>, <span class="number">7</span>, <span class="number">14</span>, <span class="number">20</span>, <span class="number">4</span>, <span class="number">4</span>, <span class="number">19</span>, <span class="number">8</span>, <span class="number">18</span>, <span class="number">20</span>, <span class="number">24</span>,
|
||
<span class="number">1</span>, <span class="number">25</span>, <span class="number">19</span>, <span class="number">58</span>, <span class="number">29</span>, <span class="number">10</span>, <span class="number">5</span>, <span class="number">15</span>, <span class="number">20</span>, <span class="number">2</span>, <span class="number">2</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">3</span>, <span class="number">5</span>,
|
||
<span class="number">51</span>, <span class="number">11</span>, <span class="number">4</span>, <span class="number">53</span>, <span class="number">23</span>, <span class="number">39</span>, <span class="number">6</span>, <span class="number">4</span>, <span class="number">13</span>, <span class="number">81</span>, <span class="number">4</span>, <span class="number">186</span>, <span class="number">5</span>, <span class="number">67</span>, <span class="number">3</span>, <span class="number">2</span>, <span class="number">15</span>,
|
||
<span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">5</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">0</span>,
|
||
<span class="number">12</span>, <span class="number">2</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">6</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">9</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">0</span>,
|
||
<span class="number">2</span>, <span class="number">2</span>, <span class="number">4</span>, <span class="number">4</span>, <span class="number">11</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">3</span>, <span class="number">6</span>, <span class="number">9</span>, <span class="number">4</span>, <span class="number">5</span>,
|
||
<span class="number">46</span>, <span class="number">18</span>, <span class="number">8</span>, <span class="number">18</span>, <span class="number">17</span>, <span class="number">3</span>, <span class="number">8</span>, <span class="number">20</span>, <span class="number">16</span>, <span class="number">10</span>, <span class="number">3</span>, <span class="number">7</span>, <span class="number">175</span>, <span class="number">4</span>, <span class="number">6</span>, <span class="number">7</span>, <span class="number">13</span>,
|
||
<span class="number">3</span>, <span class="number">7</span>, <span class="number">3</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">3</span>, <span class="number">10</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">5</span>, <span class="number">2</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">2</span>,
|
||
<span class="number">16</span>, <span class="number">3</span>, <span class="number">5</span>, <span class="number">1</span>, <span class="number">6</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">58</span>, <span class="number">20</span>, <span class="number">3</span>, <span class="number">14</span>, <span class="number">12</span>, <span class="number">2</span>, <span class="number">1</span>, <span class="number">3</span>, <span class="number">16</span>, <span class="number">3</span>, <span class="number">5</span>,
|
||
<span class="number">8</span>, <span class="number">3</span>, <span class="number">1</span>, <span class="number">8</span>, <span class="number">6</span>, <span class="number">17</span>, <span class="number">6</span>, <span class="number">5</span>, <span class="number">3</span>, <span class="number">8</span>, <span class="number">6</span>, <span class="number">13</span>, <span class="number">175</span>,
|
||
];
|
||
TABLE[byte <span class="kw">as </span>usize]
|
||
}
|
||
}
|
||
<span class="comment">// Create a new finder with the custom heuristic.
|
||
</span><span class="kw">let </span>finder = FinderBuilder::new()
|
||
.build_forward_with_ranker(Binary, <span class="string">b"\x00\x00\xdd\xdd"</span>);
|
||
<span class="comment">// Find needle with custom heuristic.
|
||
</span><span class="macro">assert!</span>(finder.find(<span class="string">b"\x00\x00\x00\xdd\xdd"</span>).is_some());</code></pre></div>
|
||
</div></details><h2 id="required-methods" class="section-header">Required Methods<a href="#required-methods" class="anchor">§</a></h2><div class="methods"><details class="toggle method-toggle" open><summary><section id="tymethod.rank" class="method"><a class="src rightside" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#317">source</a><h4 class="code-header">fn <a href="#tymethod.rank" class="fn">rank</a>(&self, byte: <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.u8.html">u8</a>) -> <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.u8.html">u8</a></h4></section></summary><div class="docblock"><p>Return the heuristic frequency rank of the given byte. A lower rank
|
||
means the byte is believed to occur less frequently in the haystack.</p>
|
||
<p>Some uses of this heuristic may treat arbitrary absolute rank values as
|
||
significant. For example, an implementation detail in this crate may
|
||
determine that heuristic prefilters are inappropriate if every byte in
|
||
the needle has a “high” rank.</p>
|
||
</div></details></div><h2 id="foreign-impls" class="section-header">Implementations on Foreign Types<a href="#foreign-impls" class="anchor">§</a></h2><details class="toggle implementors-toggle"><summary><section id="impl-HeuristicFrequencyRank-for-%26R" class="impl"><a class="src rightside" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#331-338">source</a><a href="#impl-HeuristicFrequencyRank-for-%26R" class="anchor">§</a><h3 class="code-header">impl<'a, R> <a class="trait" href="trait.HeuristicFrequencyRank.html" title="trait memchr::arch::all::packedpair::HeuristicFrequencyRank">HeuristicFrequencyRank</a> for <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.reference.html">&'a R</a><div class="where">where
|
||
R: <a class="trait" href="trait.HeuristicFrequencyRank.html" title="trait memchr::arch::all::packedpair::HeuristicFrequencyRank">HeuristicFrequencyRank</a>,</div></h3></section></summary><div class="docblock"><p>This permits passing any implementation of <code>HeuristicFrequencyRank</code> as a
|
||
borrowed version of itself.</p>
|
||
</div><div class="impl-items"><section id="method.rank" class="method trait-impl"><a class="src rightside" href="../../../../src/memchr/arch/all/packedpair/mod.rs.html#335-337">source</a><a href="#method.rank" class="anchor">§</a><h4 class="code-header">fn <a href="#tymethod.rank" class="fn">rank</a>(&self, byte: <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.u8.html">u8</a>) -> <a class="primitive" href="https://doc.rust-lang.org/1.76.0/std/primitive.u8.html">u8</a></h4></section></div></details><h2 id="implementors" class="section-header">Implementors<a href="#implementors" class="anchor">§</a></h2><div id="implementors-list"></div><script src="../../../../trait.impl/memchr/arch/all/packedpair/trait.HeuristicFrequencyRank.js" async></script></section></div></main></body></html> |