<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Tevinch's Data Notes]]></title><description><![CDATA[Tevinch's Data Notes]]></description><link>https://tevinch.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sun, 20 Sep 2026 18:22:07 GMT</lastBuildDate><atom:link href="https://tevinch.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Preserve decimal commas when pasting into Streamlit]]></title><description><![CDATA[When someone copies 23,4 from a spreadsheet, they usually expect to keep that value. If an input column interprets the comma as a thousands separator, the result can become 234 before the application ]]></description><link>https://tevinch.hashnode.dev/preserve-decimal-commas-when-pasting-into-streamlit</link><guid isPermaLink="true">https://tevinch.hashnode.dev/preserve-decimal-commas-when-pasting-into-streamlit</guid><category><![CDATA[Python]]></category><category><![CDATA[streamlit]]></category><category><![CDATA[data processing]]></category><dc:creator><![CDATA[Tevinch]]></dc:creator><pubDate>Wed, 09 Sep 2026 09:12:01 GMT</pubDate><content:encoded><![CDATA[<p>When someone copies <code>23,4</code> from a spreadsheet, they usually expect to keep that value. If an input column interprets the comma as a thousands separator, the result can become <code>234</code> before the application gets a chance to validate it.</p>
<p>That exact symptom appears in <a href="https://github.com/streamlit/streamlit/issues/7866">Streamlit issue #7866</a>. A <a href="https://github.com/streamlit/streamlit/issues/9796">later report</a> from the same reporter was closed as overlapping, so it should not be counted as an independent report or a separate confirmed bug. The useful lesson is to decide what the source punctuation means before converting the input.</p>
<p>Here is a small text-column workflow for apps that can let users paste text first and review converted values afterward. It uses Python's standard-library <code>Decimal</code> and native Streamlit widgets. The complete code below is free to use under the <a href="https://github.com/tevinch/data-shape-kit/blob/b3a37414efd449bcebded6bb26cb20f9c1e9c597/examples/streamlit-decimal-paste/LICENSE">MIT license</a>.</p>
<h2>Keep the input and the number separately</h2>
<p>Three decisions make this workflow predictable:</p>
<ul>
<li>Preserve the raw string, including leading integer zeros, alongside the result.</li>
<li>Choose a specific input grammar. This first example accepts comma decimals and does <strong>not</strong> accept thousands grouping.</li>
<li>Construct <code>Decimal</code> directly from the validated text. Return a string for the JSON preview so it does not turn into a binary floating-point number.</li>
</ul>
<p>Display formatting is a separate concern. The <a href="https://docs.streamlit.io/develop/api-reference/data/st.column_config/st.column_config.numbercolumn">NumberColumn documentation</a> says its formatting does not change the value returned by <code>st.data_editor</code>. A <a href="https://discuss.streamlit.io/t/columnconfig-numbercolumn-localized/95533">community discussion about localized formatting</a> also illustrates why changing a Python locale is not the same as selecting the browser's display language. Neither operation reconstructs punctuation already lost during conversion.</p>
<h2>A converter with a deliberately small grammar</h2>
<p>Save this as <code>decimal_comma.py</code>. It accepts a string or <code>None</code> from the text column:</p>
<pre><code class="language-python">import re
from decimal import Decimal

def convert(raw):
    if raw is None:
        return {"raw": raw, "status": "empty"}
    text = raw.strip(" \u00a0\u202f")
    if not text:
        return {"raw": raw, "status": "empty"}
    if len(raw) &gt; 256 or re.fullmatch(r"[+-]?[0-9]+(?:,[0-9]+)?", text) is None:
        return {"raw": raw, "status": "invalid"}
    return {"raw": raw, "decimal": str(Decimal(text.replace(",", "."))), "status": "valid"}
</code></pre>
<p>The regular expression requires ASCII digits, an optional leading sign and, if a comma is present, digits on both sides. It rejects currency symbols, percentages, exponents, underscores, tabs, newlines and non-finite values such as <code>NaN</code>. Ordinary space, non-breaking space (<code>U+00A0</code>) and narrow non-breaking space (<code>U+202F</code>) are allowed around the whole number.</p>
<p><code>1,234</code> is accepted as the decimal value <code>1.234</code>. If your source intended a grouped integer, this grammar is the wrong choice. An ambiguous string cannot tell you the user's intention on its own.</p>
<p>Nonempty input longer than 256 characters is rejected. The demo treats whitespace-only input as empty. The raw input remains in every report, including invalid and empty results.</p>
<h2>Put a text column in front of it</h2>
<p>Save this as <code>app.py</code> beside <code>decimal_comma.py</code>:</p>
<pre><code class="language-python">import streamlit as st
from decimal_comma import convert

st.title("Paste decimal commas as text")
st.caption("Comma decimals, without thousands grouping. Original values stay visible.")

edited = st.data_editor(
    {"Raw value": ["23,4", "0012,00"]},
    column_config={"Raw value": st.column_config.TextColumn("Raw value", max_chars=256)},
    num_rows="dynamic",
    key="raw_values",
)
if st.button("Convert"):
    st.json([convert(raw) for raw in edited["Raw value"]])
</code></pre>
<p>Use Python 3.11 or newer. Create an environment, install the version used for this example, and start the local app:</p>
<pre><code class="language-sh">python3 -m venv .venv
. .venv/bin/activate
python -m pip install streamlit==1.63.0
python -m streamlit run app.py --server.address 127.0.0.1 --server.headless true --browser.gatherUsageStats false
</code></pre>
<p>On Windows, activate with <code>.venv\Scripts\activate</code> instead. Open the local URL printed in the terminal. Click the first cell and paste this single column, then finish editing and click <strong>Convert</strong>:</p>
<pre><code class="language-text">23,4
0012,00
1.234,50
9007199254740993,50
</code></pre>
<p>The expected results are <code>23.4</code>, <code>12.00</code>, an invalid row, and <code>9007199254740993.50</code>. The third value uses dot grouping, which this minimal converter deliberately rejects. Add an empty row to see a separate <code>empty</code> result.</p>
<p>The JSON preview keeps decimals as strings. The numerical value of <code>0012,00</code> is represented as <code>12.00</code>; its leading integer zeros survive in the accompanying raw string. For arithmetic in your application, use <code>Decimal</code> objects and an appropriate arithmetic context. Python's <a href="https://docs.python.org/3/library/decimal.html">Decimal documentation</a> explains the distinction between exact construction from text and later arithmetic, which can round according to that context.</p>
<p>This app sends the browser input to the Python process on your computer. It is not a browser-only converter. The example does not save the values or submit them to another service. A remote deployment would receive those values on the remote server instead.</p>
<h2>Check the cases that are easy to miss</h2>
<p>Save this as <code>check.py</code> beside the converter, then run <code>python check.py</code>. No Streamlit installation is required for these checks:</p>
<pre><code class="language-python">from decimal_comma import convert

assert convert("23,4")["decimal"] == "23.4"
assert convert("0012,00") == {"raw": "0012,00", "decimal": "12.00", "status": "valid"}
assert convert("1,234")["decimal"] == "1.234"
assert convert("1.234,50")["status"] == "invalid"
assert convert("9007199254740993,50")["decimal"] == "9007199254740993.50"
assert convert("NaN")["status"] == "invalid"
assert convert(None) == {"raw": None, "status": "empty"}
assert convert(" \u00a0")["status"] == "empty"
print("All eight decimal checks passed.")
</code></pre>
<p>These cases check parsing and the report representation. They do not replace a real paste test from the spreadsheet and browser your users have. I checked the native text-column workflow in Chrome with Streamlit 1.63.0, including multi-row paste, empty rows, invalid grouping and long values. A text column is an alternative application input path; this does not patch <code>NumberColumn</code> or recover input already changed upstream.</p>
<h2>When your source includes thousands grouping</h2>
<p>For an application that needs both <code>1.234,50</code> and <code>1,234.50</code>, expose the decimal and grouping choices explicitly. I maintain a free <a href="https://github.com/tevinch/data-shape-kit/tree/b3a37414efd449bcebded6bb26cb20f9c1e9c597/examples/streamlit-decimal-paste">parser and runnable Streamlit example</a> with those controls.</p>
<p>Copy <code>decimal_text.py</code> and <code>LICENSE</code> from that directory beside your code, then use:</p>
<pre><code class="language-python">from decimal_text import parse_decimal_text

comma_value = parse_decimal_text("1.234,50", decimal_mark=",", group_mark=".")
dot_value = parse_decimal_text("1,234.50", decimal_mark=".", group_mark=",")
assert str(comma_value) == "1234.50"
assert str(dot_value) == "1234.50"
</code></pre>
<p>That helper validates groups of three digits, supports three explicit space characters, rejects malformed groups such as <code>12.34,5</code>, and returns <code>Decimal</code> objects. It has 12 tests covering its documented grammar. The accompanying app keeps raw values visible and clears the previous conversion report when the table or selected format changes.</p>
<p>Choose one source format for a conversion batch. If rows mix formats, ask the user to separate them or provide per-row format information. Guessing from punctuation can produce a plausible but incorrect number.</p>
<h2>Buy me a coffee, if this helped</h2>
<p>If this saved you a little time, you're welcome to buy me a coffee. Please don't feel obliged — using the example, reporting an issue or sharing it is appreciated too.</p>
<ul>
<li><strong>USDC on Solana:</strong> <code>9tY6D9mwcFaJwwzEHvw2v7nhSpdjqjNBYtuooyBN6rYy</code></li>
<li><strong>USDC on Base:</strong> <code>0x568Ab98578d682FB0B0b45619BE73EbFfbf5a6eA</code></li>
</ul>
<p>Please match the asset and network exactly. Fees depend on your wallet or exchange. Thank you! — Tevinch</p>
]]></content:encoded></item><item><title><![CDATA[Why splitting spreadsheet clipboard text on newlines breaks pasted tables]]></title><description><![CDATA[A spreadsheet cell can contain a newline. A selected range can also end with empty cells. Both are easy to lose when a web form treats pasted text as text.trim().split('\n') and then splits each line ]]></description><link>https://tevinch.hashnode.dev/why-splitting-spreadsheet-clipboard-text-on-newlines-breaks-pasted-tables</link><guid isPermaLink="true">https://tevinch.hashnode.dev/why-splitting-spreadsheet-clipboard-text-on-newlines-breaks-pasted-tables</guid><category><![CDATA[JavaScript]]></category><category><![CDATA[Web Development]]></category><dc:creator><![CDATA[Tevinch]]></dc:creator><pubDate>Wed, 09 Sep 2026 07:53:53 GMT</pubDate><content:encoded><![CDATA[<p>A spreadsheet cell can contain a newline. A selected range can also end with empty cells. Both are easy to lose when a web form treats pasted text as <code>text.trim().split('\n')</code> and then splits each line on tabs.</p>
<p>Here are three small regression cases for a spreadsheet paste field, followed by a plain JavaScript integration. The examples use <a href="https://github.com/tevinch/data-shape-kit/tree/main/javascript/clipboard-table">Clipboard Table</a>, a free MIT-licensed module I maintain. It reads a defined quoted-TSV format; it does not read spreadsheet files.</p>
<h2>1. A newline does not always end a row</h2>
<p>This synthetic payload has a header and one data row. The Notes cell contains two lines:</p>
<pre><code class="language-js">const text = 'SKU\tNotes\r\n00123\t"First line\nSecond line"\r\n';
</code></pre>
<p>Splitting at every newline loses the distinction between a record boundary and a line break inside a cell. A quoted-TSV parser needs to remember whether it is inside a quoted field:</p>
<ul>
<li>Outside a quoted field, a tab ends a cell and a record separator ends a row.</li>
<li>Inside a quoted field, tabs and line breaks are part of the value.</li>
<li>Two consecutive quotes inside a quoted field represent one literal quote.</li>
</ul>
<p>This matters in real integrations: community reports describe <a href="https://stackoverflow.com/questions/67773475/copy-paste-from-excel-difference-between-line-break-in-cell-and-row-delimiter">line breaks inside copied Excel cells</a> and <a href="https://github.com/handsontable/handsontable/issues/8117">multiline Google Sheets paste behavior</a>. They illustrate failure cases, not a claim that those projects still have unresolved bugs. HTML clipboard paths can behave differently from the plain-text path used here.</p>
<p>To run the examples below, download the <a href="https://github.com/tevinch/data-shape-kit/raw/660592d4ef55b42da4c2dc1561d658c7b92456d3/downloads/clipboard-table-v0.1.0.zip">module archive</a>, extract it, and save the code as <code>check.mjs</code> beside <code>index.mjs</code>. Run <code>node check.mjs</code> with Node.js 22 or newer.</p>
<pre><code class="language-js">import assert from 'node:assert/strict';
import { parseClipboard, toRecords } from './index.mjs';

const text = 'SKU\tNotes\r\n00123\t"First line\nSecond line"\r\n';

assert.deepEqual(toRecords(parseClipboard(text)), [
  { SKU: '00123', Notes: 'First line\nSecond line' }
]);
</code></pre>
<p>The identifier remains a string. If the source application has already removed its zeros or rounded a long number, a text parser cannot reconstruct the missing information.</p>
<h2>2. Trimming can delete a column</h2>
<p>Append these checks to the same file:</p>
<pre><code class="language-js">assert.deepEqual(parseClipboard('\t00123\t\r\n'), [
  ['', '00123', '']
]);

assert.deepEqual(parseClipboard('A\r\n\r\n'), [
  ['A'], ['']
]);

assert.deepEqual(parseClipboard('"He said ""hello"""\t"x\ty"'), [
  ['He said "hello"', 'x\ty']
]);
</code></pre>
<p>Tabs at the edges encode empty cells. Calling <code>trim()</code> first would erase them. The parser also preserves spaces inside values.</p>
<p>The second check makes a deliberate boundary rule visible: one terminal record separator ends the preceding row; an additional separator represents an explicit blank row. Your application's policy might differ, but make it explicit and test it before dropping blank rows.</p>
<h2>3. Valid rows do not always make valid objects</h2>
<p>Two columns named <code>SKU</code> cannot both map to the same object property without losing one value. Parsing and object conversion are separate steps:</p>
<pre><code class="language-js">assert.throws(
  () =&gt; toRecords(parseClipboard('SKU\tSKU\r\n00123\t00456')),
  error =&gt; error.code === 'DUPLICATE_HEADER'
);

assert.throws(
  () =&gt; toRecords(parseClipboard('SKU\tNotes\r\n00123')),
  error =&gt; error.code === 'RAGGED_ROW'
);

console.log('All six clipboard checks passed.');
</code></pre>
<p><code>parseClipboard</code> preserves ragged rows. <code>toRecords</code> requires exact, unique, nonblank headers and rows of matching width. It does not trim or rename headers. Keep the array representation if your application needs to resolve those cases interactively.</p>
<h2>Connect a paste field</h2>
<p>For a page with <code>&lt;textarea id="paste-input"&gt;&lt;/textarea&gt;</code> and <code>&lt;pre id="result"&gt;&lt;/pre&gt;</code>, serve <code>index.mjs</code> beside the page and put this in a module script:</p>
<pre><code class="language-js">import { parseClipboard } from './index.mjs';

const input = document.querySelector('#paste-input');
const output = document.querySelector('#result');

input.addEventListener('paste', event =&gt; {
  if (!event.clipboardData?.types.includes('text/plain')) return;
  event.preventDefault();

  try {
    const rows = parseClipboard(
      event.clipboardData.getData('text/plain'),
      { maxChars: 500_000, maxRows: 2_000, maxColumns: 80 }
    );
    output.textContent = JSON.stringify(rows, null, 2);
  } catch (error) {
    output.textContent = error.message;
  }
});
</code></pre>
<p>The handler reads only the current paste event's plain text and replaces the preview on success or failure. Use <code>textContent</code> so a cell containing HTML is displayed as text. The parser's <code>maxChars</code> limit counts UTF-16 code units, matching JavaScript's string length. Row and column limits apply to the parsed table.</p>
<p>MDN documents the event's <a href="https://developer.mozilla.org/en-US/docs/Web/API/ClipboardEvent/clipboardData">clipboardData property</a> and <a href="https://developer.mozilla.org/en-US/docs/Web/API/Element/paste_event">preventing default paste behavior</a>. This example displays the parsed result; inserting cells into your grid, preserving selection and handling concurrent edits belong in your application's update layer.</p>
<h2>Try it without setting up a project</h2>
<p>The <a href="https://github.com/tevinch/data-shape-kit/releases/tag/clipboard-table-playground-v0.1.0">local browser playground</a> is a ZIP containing a self-contained page. Extract it, open <code>index.html</code>, load the example or paste a range, and inspect arrays or header-based JSON objects. The page makes no uploads and needs no external runtime assets. Its preview shows up to 30 data rows and 8 columns; the JSON result includes the complete parsed table within the configured limits.</p>
<p>The <a href="https://github.com/tevinch/data-shape-kit/tree/main/javascript/clipboard-table">source and API guide</a> explain the exact grammar and include 31 parser/API tests with 144 deterministic round trips. Those tests verify text handling; they are not a compatibility matrix for every spreadsheet and browser version. Check a real copy/paste from the applications your users have.</p>
<p>This module does not interpret HTML clipboard data, read XLSX files, preserve styles or merged cells, or neutralize spreadsheet formulas. Its serializer preserves formula-looking strings, which a receiving spreadsheet may evaluate. If you already have a CSV/TSV parser, check its quoting, empty-row and type-conversion settings before adding another dependency.</p>
<h2>Buy me a coffee, if this helped</h2>
<p>If this saved you a little time, you're welcome to buy me a coffee. Please don't feel obliged — using the code, reporting an issue, or sharing it is appreciated too.</p>
<ul>
<li><strong>USDC on Solana:</strong> <code>9tY6D9mwcFaJwwzEHvw2v7nhSpdjqjNBYtuooyBN6rYy</code></li>
<li><strong>USDC on Base:</strong> <code>0x568Ab98578d682FB0B0b45619BE73EbFfbf5a6eA</code></li>
</ul>
<p>Please match the asset and network exactly. Fees depend on your wallet or exchange. Thank you! — Tevinch</p>
]]></content:encoded></item></channel></rss>