Skip to content

Feature/matrix wide scaling - #351

Merged
tremblap merged 6 commits into
flucoma:mainfrom
tremblap:feature/matrix-wide-scaling
Nov 14, 2026
Merged

tremblap merged 6 commits into
flucoma:mainfrom
tremblap:feature/matrix-wide-scaling

Conversation

@tremblap

Copy link
Copy Markdown
Member

Mostly for 2D drawing, we need matrix-wide, not column-wise, data processing to preserve the original shape and the relevance of diagonal distances. This is done for the 3 transformers.

At the moment, there are 3 questions for @weefuzzy @jamesb93 and @tedmoore :

  1. It currently works as an overide of the original code (to test the algo, and review my implementation idea of keeping the processors intact post fitting (so still processing by vectors) - I think it is a good idea, but someone might disagree.
  2. at the moment, the idea is to have a flag to process colwise (as before, and would be the default) or matrixwise. this implies a forking only in the algo, and I could code it, except if I'm told we should do a 'fit-full-dataset' method instead. both are trivial to implement and I'll do it in the next push on this PR if this is the agreed way
  3. if this is the agreed way, the isDSwide name is appalling. Even just dswide is appalling. but I am struggling to find something for now that stays in the vocabulary (byDim, byDS, byCol, colwise, keepshape, relative) so please let's brainstorm. obviously the default will be the backward-compatible switch

Thanks!

@tremblap
tremblap marked this pull request as draft October 27, 2026 13:41
@tremblap

Copy link
Copy Markdown
Member Author

example code in SC to see it work and test the dims are right (3x2 is perfect for debugging :) )

d = FluidDataSet(s).load(Dictionary.newFrom([\cols, 2, \data, Dictionary.newFrom([\p1, [0.1, 0.2], \p2, [0.3, 0.9], \p3, [0.2, 0.5]])]))
d.print

~fp = FluidPlotter(bounds:Rect(200,200,600,600), xmin: -0.1, xmax: 1.1, ymin: -0.1, ymax: 1.1)

d.dump{|x|defer{~fp.dict = x}}

n = FluidNormalize(s)

n.fitTransform(d,d)

r = FluidRobustScale(s)

r.fitTransform(d,d)

w = FluidStandardize(s)

w.fitTransform(d,d)

~fp.xmin_(-3).ymin_(-3).xmax_(3).ymax_(3)

@tedmoore

Copy link
Copy Markdown
Member

My first thought for the arg name was "by column" (or byCol or whatever is appropriate), which was also one of your suggestions, so there's some resonance there.

@jamesb93

jamesb93 commented Nov 3, 2026

Copy link
Copy Markdown
Member

PA explained it to me. It's been a while since I've used FluCoMa closely so I needed the crash course. It all makes sense, and I think byCol is a good shout. Normalising byCol has a nice self-documenting vibe to it.

@tremblap

tremblap commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

now with the right name, and a fork in 'normalize' for @weefuzzy to review

and a max patch to make it simple:


----------begin_max5_patcher----------
732.3ocwWF0biBBDG+YymBGdNWFQMp49pbSmLnRxQGEbPrWx0o8y9AKRp2kz
FlDauGRXbWj+7CV1EedQ.pTbf1iB+d3OBCBddQP.XxXHX74.TK4PUCoG5Fpk
12S1SQKs9TzCJvd4wJQSXzxvcLkRR386Dx1PFuaPEJFT5F2qvGZ0FZnJX.wi
VY0vvHJe7awottZ6m5XG0NEQnvGFc0QTU+jw2uURqTVuoQEqzS.75MPSNzDG
uJ5zKoklwcJGar8xhEl+VNKzimC5wE2F83M4.1Yq+xoudnsye9xtQ9xAvhSy
LMIeB38X+.6xXDeALNEj1QjjVphJ2R4jxF.lnQekBYMUN0x4Pu7JfmLtUtw1
jlep4zasi0n0tEFRztlAV8ptFgROmPy0hCm9KM1ms0K07PCe+.fKsxEit80h
hDfdLbRO6SHHnbPoDb+ilw9DFbFnkD9deh1SRsa1t+uLm34a+rSR6n75vFAo
16Eg765DcArKVj9w6kyHi0rp2Ic75yY68yFaFFlfSjGgv1o+7f5Lf2L7UhfG
MVSTjQPGI8us4LpUDakIZktZTzpX23ZbE6bkXbsYpqDmqXiq0gOX8nWqGUSW
e6syTmri5IOQq2pWnz7sknq8wzGfr2nX5bk1VRqmD8DfZn6IUGmbH4COAM0Y
Koq6j6wYy7EbXydx0kuIMreSe06zZI2dVMWoMaVsMIeYmDrvZBj5opW+vapb
Ahiuehygh4Xb9+Kjgqn4eoq6lXL1d4Tbj+DC8.0v3+6M0AYL1+6kgdwfrxMw
bWnN7Mgpo8JFmXRdMoSI19bwUZuEROF3qnio7+cKj4RxeIDYts5UEZNHJwCc
bSl6inXeTJcNTB6gREyfP49PzLniOAc4moi83qtn0STY+XmAIz4qdT.ebPwR
3QF29XF7nj9Dy0e3hIHhTmUQoSoLHsW0+Pl8qPPsB8mYv0e9hM0ATdFA4BMe
VPeGwxAjxbwKK9CoA8JG
-----------end_max5_patcher-----------

@weefuzzy

weefuzzy commented Nov 3, 2026

Copy link
Copy Markdown
Member

My two in-an-ideal-world comments would be

  • considering merits of a numpy-style axis argument instead that would allow to process colwise (features), rowwise (samples) or all-in
  • not being mad keen on the inelegance of the branching

But seeing as I don't have current time / bandwidth to think properly about the added ramifications of the first or come up with something more elegant for the second, I say have at it.

@tremblap

tremblap commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

@weefuzzy axis 0 for all, axis 1 and 2 for either, is easy enough for me to implement (and we can refactor the fork eventually)

This also has the advantage of allowing for eventual other dimensions (aka time in dataseries).

@weefuzzy

weefuzzy commented Nov 3, 2026

Copy link
Copy Markdown
Member

Well, -1 for all, 0 for rows (samples) and 1 for cols (features) would be more pythonic.

But, I think there's possible non-obvious side-effects on other bits of the code for those objects that would need be thought about. Hence, if you just want to solve the problem in front of you quickly, do as you were.

@tremblap

tremblap commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

ok and HCI-devil-advocate here: if we want to merge now, I can do axis -1 and 0 and leave it at that, and then on the desiderata list, we put 1 and 2 later. a bit like the MFCC @startcoeff which might, one day, get something else than 0 and 1 🤣

the advantage of @axis is that we have that in other objects, and -1 for all is good as we are 0 counting everywhere - so changing that little interface convention now prevent a future breaking change?

@tedmoore

tedmoore commented Nov 3, 2026

Copy link
Copy Markdown
Member

I like the numpy-esque axis.

@tremblap

tremblap commented Nov 3, 2026 •

Copy link
Copy Markdown
Member Author

and I've checked (https://numpy.org/doc/2.2/reference/generated/numpy.mean.html), axis 0 in numpy is what we currently do (column-wise) - I'll sleep on it, it feels less 'self-explanatory' as @jamesb93 so kindly qualified 'byCol' but then, we open up a bit more IT knowhow - a balancing act.

@tedmoore

tedmoore commented Nov 3, 2026

Copy link
Copy Markdown
Member

One thought I had re: byCol is that if I specify 1 (assuming that means true), then I might guess that 0 would mean the "opposite": byRow, which if I'm understanding the above conversation correctly would be a wrong guess...

@tremblap

tremblap commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

yeah I got squishy during my post-coding thinking about it 🤣 indeed axis with -1 and 0 is a good future proof way I think

@tremblap
tremblap marked this pull request as ready for review November 14, 2026 14:59
@tremblap
tremblap merged commit 2770434 into flucoma:main Nov 14, 2026
3 checks passed
@tremblap
tremblap deleted the feature/matrix-wide-scaling branch November 14, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants