New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flash-Attn 3的支持 #308

Open

echo-valor opened this issue Aug 1, 2024 · 1 comment

echo-valor commented Aug 1, 2024

打扰，请问PAI-Megatron何时支持Flash-attn 3？

Collaborator

jerryli1981 commented Aug 6, 2024

您好，在即将推出的基于Mcore的llama3.1中，我们将对Flash Attention 3进行测试并尝试支持

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment