Using xfce4-panel from git://git.xfce.org/xfce/xfce4-panel 4.13.3git-a8a3b834 Sizing of the panel increments in 10 px rather than 2 px as previous setting. Can manually set it in 2 px increments using Settings Editor and it's possible, though fiddly to use a mouse on the panel size slider to move it.
Weird, I can't reproduce with current master, it steps 1 by 1 using both touchpad two-finger scroll or keyboard arrows. But I remember it used to increment in 2px steps.
How did you change the size? I get 10px changes only when scrolling over the GtkScale widget (paging), with my up/down arrow keys I get 1px changes.
scrolling in panel settings no idea how you up/down arrow it ...
You can either move the focus to that widget with tabbing, or you can click the slider handle once to give it focus. Then your keys should work nicely with the GtkScale