Research update · CEC-2013 LSGO, D=1000

Subspace optimization is a specialist, not a general improvement

Measured against plain Differential Evolution, the dual-EA subspace method loses on most problem classes and wins by roughly 50× on overlapping structure. That pattern is the finding. It tells us which problems the method is actually for.

Where it helps

Every run below is 3 million evaluations at D=1000. The parent is plain DE with no projection and static parameters (F=0.5, CR=0.9); the subspace arm is the dual-EA with LoRA rank 1, same optimizer, same budget, same seeds.

Aggregated by structural class

Is it this map, or any map of the same size?

All three projections searched at the same 64 numbers, same dual-EA scheme, same budget and seeds. If any reduction of that size performed alike, the structure of the map would be irrelevant and there would be nothing to study.

The map matters: at identical search dimension the three are not interchangeable.

Convergence

Does self-adapting the optimizer's parameters help?

PyMOO can evolve DE's F and CR per individual. Matched pairs, same seeds and budget, static against self-adaptive:

Does learning a projection parameter help?

We made the projection's output scale a searched decision variable. That costs one extra dimension, and the neutral value reproduces the unscaled map exactly. It addresses a defect we measured: LoRA's factors overshoot the box by about 100×, which clips roughly 95% of coordinates.

The gain tracks how much work the projection is doing. Random projection, which must produce the whole solution, improves in every pair. The dual-EA, which is anchored on the full-space best and so already has its reach handled, barely moves.

Is the headline result real? A permutation control

Shuffling the variable order gives back the same function, with the same weights and the same difficulty. All it breaks is any accidental alignment between the benchmark's coordinate ordering and the method's internal reshape. We ran it over all 15 functions and 3 seeds:

Optimising the optimizer

A natural extension of the two-population idea is to let one population optimise the optimizer's own parameters rather than more solutions. We implemented that: a second population whose genome is (F, CR), scored by the progress the solution population actually makes under those values.

It does not pay at this budget. Racing λ candidate parameter sets to advance one costs a λ-fold tax on progress, and the better parameters it finds do not repay that. Over plain DE it landed behind static parameters. Over the dual-EA it beat the evolving baseline in only 3 of 9 runs. We are reporting it as a negative result.

Does it hold at ten times the dimension?

One seed at D=10000, DE. Note this is a different problem family rather than a harder version of the same one: at D≠1000 the benchmark generates its structure from a seed instead of loading the official data files, so these are comparable to each other but not to the D=1000 numbers.

The ordering is unchanged: the dual-EA variants lead, fixed projections trail badly.

What the controls rule out

F1 is an artifact. Its weights factor as 10^(c(32i+j)) = 10^(32ci) × 10^(cj), which is exactly the multiplication table the LoRA map produces. It is rank-1 to machine precision. A permutation control over all 15 functions and 3 seeds showed F1 collapsing by ~10¹² when variables are shuffled, while nothing else moved. F1 is therefore best treated as a special case rather than evidence of a general effect.

Fixed projections are worse than no projection. Searching a fixed LoRA or random projection in absolute mode lands five to six orders behind plain DE. Clipping to the box turns a rank-4 solution into a full-rank saturated point, destroying the structure the method depends on.

Optimising the optimizer did not pay at this budget. Racing candidate parameter sets costs more evaluations than the better parameters return. Worth revisiting at larger budgets or with a cheaper credit-assignment scheme.