DoubleColumn.set only replaces first 0 when using isEqualTo(0.0)

While following the plotting examples I did the following (in Scala):

    val property: Table = Table.read().csv("./data/tablesaw/sacramento_real_estate_transactions.csv")
    val sqft: DoubleColumn = property.numberColumn("sq__ft").asDoubleColumn()
    //sqft.set(sqft.isEqualTo(0.0), java.lang.Double.NaN)
    sqft.set(sqft.isEqualTo(0.0), -9999.0)
    println("Unique +++++++++++++++++++++++++++")
    val tmp = sqft.unique()
    tmp.sortAscending()
    println(tmp.asObjectArray().mkString(","))

    val histogram2 = Histogram.create("Distribution of property sizes", property, "sq__ft")
    Plot.show( histogram2 )

I noticed that the plot still had values in the 0-199 range (unlike the guide’s output). To check this I looked at the CSV file and confirmed the 0 were exact. I also print the data of the unique values but it shows (see below) shows that the 0.0 are repeated. The repeats are also an issue and are reported in #586.

However as seen below the set methods only seems to substitute the first 0 value.

Unique +++++++++++++++++++++++++++
-9999.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,484.0,539.0,588.0,610.0,611.0,623.0,625.0,682.0,696.0,722.0,723.0,746.0,756.0,760.0,779.0,780.0,795.0,796.0,797.0,800.0,804.0,810.0,832.0,834.0,836.0,838.0,840.0,844.0,846.0,852.0,861.0,864.0,868.0,871.0,876.0,881.0,884.0,888.0,900.0,901.0,904.0,906.0,909.0,911.0,918.0,924.0,930.0,932.0,933.0,936.0,941.0,948.0,950.0,952.0,956.0,957.0,958.0,960.0,962.0,963.0,964.0,966.0,967.0,970.0,972.0,975.0,984.0,988.0,990.0,994.0,998.0,1000.0,1004.0,1006.0,1007.0,1009.0,1010.0,1011.0,1012.0,1013.0,1014.0,1018.0,1019.0,1020.0,1022.0,1029.0,1032.0,1039.0,1040.0,1043.0,1050.0,1051.0,1056.0,1058.0,1060.0,1064.0,1067.0,1073.0,1074.0,1075.0,1080.0,1082.0,1088.0,1089.0,1092.0,1093.0,1094.0,1098.0,1099.0,1100.0,1103.0,1104.0,1108.0,1110.0,1112.0,1115.0,1116.0,1117.0,1118.0,1119.0,1120.0,1122.0,1123.0,1124.0,1126.0,1127.0,1130.0,1132.0,1134.0,1137.0,1139.0,1140.0,1144.0,1146.0,1150.0,1152.0,1154.0,1156.0,1157.0,1158.0,1159.0,1160.0,1161.0,1162.0,1164.0,1166.0,1167.0,1169.0,1170.0,1172.0,1174.0,1175.0,1176.0,1177.0,1179.0,1182.0,1183.0,1185.0,1187.0,1188.0,1189.0,1193.0,1196.0,1199.0,1200.0,1202.0,1204.0,1206.0,1207.0,1209.0,1211.0,1212.0,1215.0,1216.0,1217.0,1219.0,1220.0,1229.0,1231.0,1232.0,1235.0,1240.0,1245.0,1247.0,1248.0,1249.0,1250.0,1252.0,1253.0,1255.0,1258.0,1260.0,1261.0,1262.0,1264.0,1265.0,1266.0,1269.0,1272.0,1273.0,1276.0,1277.0,1280.0,1284.0,1285.0,1287.0,1289.0,1291.0,1292.0,1294.0,1296.0,1300.0,1302.0,1304.0,1305.0,1306.0,1309.0,1310.0,1315.0,1316.0,1317.0,1319.0,1320.0,1321.0,1324.0,1326.0,1327.0,1328.0,1329.0,1331.0,1337.0,1338.0,1340.0,1341.0,1343.0,1344.0,1348.0,1351.0,1353.0,1354.0,1356.0,1358.0,1360.0,1362.0,1364.0,1366.0,1367.0,1370.0,1371.0,1373.0,1375.0,1376.0,1380.0,1381.0,1382.0,1386.0,1390.0,1392.0,1393.0,1394.0,1400.0,1401.0,1404.0,1406.0,1407.0,1410.0,1411.0,1416.0,1418.0,1419.0,1420.0,1424.0,1427.0,1428.0,1430.0,1436.0,1438.0,1439.0,1440.0,1441.0,1443.0,1446.0,1448.0,1449.0,1450.0,1451.0,1452.0,1456.0,1457.0,1462.0,1463.0,1465.0,1466.0,1468.0,1472.0,1473.0,1475.0,1477.0,1478.0,1479.0,1483.0,1485.0,1488.0,1493.0,1497.0,1498.0,1502.0,1503.0,1505.0,1506.0,1510.0,1511.0,1512.0,1513.0,1516.0,1517.0,1520.0,1522.0,1524.0,1527.0,1529.0,1531.0,1540.0,1541.0,1543.0,1548.0,1549.0,1555.0,1559.0,1566.0,1567.0,1570.0,1574.0,1577.0,1578.0,1580.0,1582.0,1586.0,1587.0,1590.0,1591.0,1593.0,1595.0,1596.0,1598.0,1601.0,1603.0,1605.0,1606.0,1608.0,1609.0,1616.0,1621.0,1623.0,1624.0,1627.0,1628.0,1636.0,1637.0,1638.0,1639.0,1643.0,1646.0,1650.0,1653.0,1655.0,1656.0,1659.0,1669.0,1670.0,1671.0,1672.0,1676.0,1677.0,1678.0,1683.0,1685.0,1686.0,1690.0,1691.0,1695.0,1697.0,1711.0,1712.0,1713.0,1714.0,1715.0,1716.0,1718.0,1720.0,1721.0,1724.0,1727.0,1735.0,1736.0,1739.0,1740.0,1744.0,1751.0,1758.0,1766.0,1768.0,1770.0,1776.0,1788.0,1789.0,1795.0,1798.0,1799.0,1800.0,1801.0,1808.0,1810.0,1811.0,1816.0,1820.0,1828.0,1829.0,1830.0,1838.0,1843.0,1844.0,1851.0,1856.0,1857.0,1859.0,1871.0,1872.0,1876.0,1882.0,1887.0,1888.0,1890.0,1892.0,1899.0,1900.0,1904.0,1905.0,1910.0,1914.0,1915.0,1917.0,1921.0,1922.0,1928.0,1929.0,1936.0,1939.0,1940.0,1943.0,1953.0,1955.0,1961.0,1962.0,1968.0,1981.0,1982.0,1991.0,1993.0,1995.0,2002.0,2004.0,2016.0,2025.0,2026.0,2030.0,2049.0,2052.0,2053.0,2054.0,2056.0,2068.0,2085.0,2093.0,2096.0,2100.0,2109.0,2110.0,2111.0,2126.0,2136.0,2142.0,2155.0,2159.0,2160.0,2161.0,2162.0,2163.0,2166.0,2169.0,2170.0,2172.0,2175.0,2185.0,2187.0,2199.0,2205.0,2208.0,2212.0,2213.0,2214.0,2218.0,2222.0,2235.0,2254.0,2258.0,2274.0,2278.0,2280.0,2295.0,2306.0,2307.0,2309.0,2325.0,2329.0,2334.0,2346.0,2347.0,2354.0,2356.0,2359.0,2367.0,2372.0,2379.0,2382.0,2442.0,2447.0,2457.0,2462.0,2475.0,2484.0,2491.0,2494.0,2503.0,2504.0,2508.0,2550.0,2555.0,2575.0,2577.0,2581.0,2592.0,2605.0,2606.0,2607.0,2652.0,2660.0,2687.0,2711.0,2724.0,2725.0,2734.0,2787.0,2789.0,2790.0,2800.0,2824.0

I assume all 0 should be cahnged. Am I correct?

Using version 0.34.2

[benmccann]

Regarding the docs, there are some alternatives we’re considering here: https://github.com/jtablesaw/tablesaw/issues/457

[hmf]

I really do apologize for the documentation being outdated, (or occasionally probably incorrect from the beginning). It throws a lot of new users off. I would love to fix it but I’m starting to think I’d need to quit my job just to keep up with issues, PRs, and coding the occasional enhancement.

Don’t apologize. You have done a great job. Set your priorities and let others contribute.

As for the documentation I have 2 projects with Scala and use MDoc and Tut respectively to process code snippets in Markdown. This has 2 advantages: a) you just copy and paste test cases to the markdown files to generate the examples and b) the code snippets will be automatically checked when compiled so its easier to maintain. Unfortunately I have not found a Java version of these. May someone else knows?

I hope tablesaw is useful to you once you get past these hurdles.

As I said, the plotting capabilities is a big plus.

I will open an issue for making missing value handling standard in plots, since as you point out, we do that for stats anyway.

Ok. Not an issue for me now. So low priority.

Thanks.