How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? | Xiaol.x | Podwise