Is it possible to replace the model's Text Encoder with other models, such as google/gemma-2-9b?

[lymanzhao]

Hi，

I'm investigating how to customize the Text Encoder part of the model. I'm wondering if it's possible to replace the original Text Encoder with other pre-trained models, such as google/gemma-2-9b.

Specifically, I have the following questions:

Is this replacement technically feasible?
If feasible, what compatibility issues should be considered?
What potential impacts on model performance might occur after the replacement?
Are there any example code snippets or tutorials available for reference?
Any related suggestions and guidance would be greatly appreciated!

Thanks