have read some higher caliber pistols can take a lower caliber barrels but not vice versa.
A .38 and a .357magnum are not different calibers. They are both the same diameter - so close anyway that it's negligible. The difference is in the case length and the powder charge. A .357magnum is basically a longer more powerful .38, requiring a longer chamber. Therefore, a .38 will fit into a .357 longer chamber but the .357magnum will not fit in the.38 shorter chamber. But they are the same caliber. The .38 came first and I think the reason they are called different names (caliber designator) is that one is measured using the full diameter of the barrel (.38 inches) and the other is measured to the inside of the grooves of the barrel (.357). It was a marketing thing when the .357 mag was introduced.
Not so with a .357sig or magnum and a .40 . They are different size in diameter and the .40 will require a bigger hole (barrel) to go thru. But the .357 is way too small for the .40 barrel.