Run llm on qualcomm 8gen4 device failed!

[jixiedaima]

The issue I update here. Not soved, but closed.
#20619

Here is my test steps:
I update the latest commit and try it again:
commit d065779 (HEAD -> main, origin/main, origin/HEAD)
Author: Nikita Lutsenko nlutsenko@meta.com
Date: Wed Jul 8 18:16:51 2026 -0700

ovr_config | build_mode | Migrate constraint refs from aliases to subtargets in xplat.

Differential Revision: D110548728

Pull Request resolved: #20783
./qnn_llama_runner --model_path Llama-3.2-3B-Instruct.SM8750.pte --tokenizer_path tokenizer.model --decoder_model_version llama3 --prompt 'hello' --seq_len 128

I get tokenizer.json and tokenizer.model from huggingface link:
Llama-3.2-3B-Instruct

When I use tokenizer.model, here is the error:
I tokenizers:regex.cpp:27] Registering override fallback regex
I 00:00:00.000986 executorch:runner.cpp:149] creating module: model_path=Llama-3.2-3B-Instruct.SM8750.pte
I 00:00:00.001009 executorch:runner.cpp:150] creating runner: tokenizer_path=tokenizer.model
I 00:00:00.001011 executorch:runner.cpp:151] eval mode=1
E tokenizers:hf_tokenizer.cpp:167] Error parsing json file: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'I'
I 00:00:00.069284 executorch:llm_runner_helper.cpp:72] Loaded TikToken tokenizer
[INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol
[INFO] [Qnn ExecuTorch]: soc_model in soc_info: SM8750
[INFO] [Qnn ExecuTorch]: backend_type: kHtpBackend
[INFO] [Qnn ExecuTorch]: library_path:
[INFO] [Qnn ExecuTorch]: dump intermediate outputs: (null)
[INFO] [Qnn ExecuTorch]: log_level:
[INFO] [Qnn ExecuTorch]: profile_level: kProfileOff
[INFO] [Qnn ExecuTorch]: the size of qnn context binary: -1841596872
[INFO] [Qnn ExecuTorch]: Is on-device graph construction: 1
[INFO] [Qnn ExecuTorch]: Enable shared buffer: 0
Segmentation fault

When I use tokenizer.json, here is the error info:
E0000 00:00:1783562121.047510 30815 re2.cc:237] Error parsing '((?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]|\s[\r\n...': invalid perl operator: (?!
I tokenizers:re2_regex.cpp:27] Re2 failed to compile regex: ((?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]|\s[\r\n]+|\s+(?!\S)|\s+), error: invalid perl operator: (?!
This may be ok if a fallback regex is used.
I tokenizers:regex_lookahead.cpp:26] Creating PCRE2 regex
I 00:00:00.349717 executorch:llm_runner_helper.cpp:55] Loaded json tokenizer
[INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol
[INFO] [Qnn ExecuTorch]: soc_model in soc_info: SM8750
[INFO] [Qnn ExecuTorch]: backend_type: kHtpBackend
[INFO] [Qnn ExecuTorch]: library_path:
[INFO] [Qnn ExecuTorch]: dump intermediate outputs: (null)
[INFO] [Qnn ExecuTorch]: log_level:
[INFO] [Qnn ExecuTorch]: profile_level: kProfileOff
[INFO] [Qnn ExecuTorch]: the size of qnn context binary: -1841596872
[INFO] [Qnn ExecuTorch]: Is on-device graph construction: 1
[INFO] [Qnn ExecuTorch]: Enable shared buffer: 0
Segmentation fault

If the test cmd is wrong, or tokenizer file it not right, please let me know. thank you~

cc @cbilgin @psiddh

[psiddh]

cc @quic-boyuc @shewu-quic @haowhsu-quic @winskuo-quic @chunit-quic

[chenweng-quic]

Hi @jixiedaima ,
Could you provide the following for us to reproduce the issue?

1. How you build executorch qnn.
2. QNN SDK version used.
3. What artifacts is pushed to device for execution.

Also, it seems like you are taking the .pte from l3utterfly/Llama-3.2-3B-Instruct-qnn-executorch Llama-3.2-3B-Instruct.SM8750.pte
(https://huggingface.co/l3utterfly/Llama-3.2-3B-Instruct-qnn-executorch/tree/main?)

Since the README doesn't show how the .pte is generated, it is hard to debug. If you have any information about the pte, could you provide that?
If you want to run llama3.2 3B using mainline executorch, refer https://github.com/pytorch/executorch/tree/main/examples/qualcomm/oss_scripts/llama#llama32