The notochord alone is common to both.
"Thomas Henry Huxley; A Sketch Of His Life And Work"
P. Chalmers Mitchell
In most modern fishes and in all higher vertebrates the notochord is gradually removed as the bodies of the vertebrae are formed about it; but in the Devonian fishes it persists through maturity and the vertebrae remain incomplete.
"The Elements of Geology"
William Harmon Norton
The skeleton is completely ossified and the notochord removed.
"The Elements of Geology"
William Harmon Norton