Skip to content
Snippets Groups Projects

Introduce Chars

Merged Björn Ludwig requested to merge introduce_chars into main
1 file
+ 3
4
Compare changes
  • Side-by-side
  • Inline
@@ -120,11 +120,10 @@ class CharProbs:
row["bigram"][0] in self.chars and row["bigram"][1] in self.chars
):
bi_probs[row["bigram"]] = absolute_count
total_sum = sum(bi_probs.values())
assert total_sum > 0
normalizer = 100 / total_sum
absolute_sum = sum(bi_probs.values())
for character, count in bi_probs.items():
bi_probs[character] = count / normalizer
bi_probs[character] = count / absolute_sum
assert round(sum(bi_probs.values()), 8) == 1
assert len(bi_probs) == reader.line_num or len(bi_probs) <= comb(
reader.line_num, 2
)
Loading