submission_id: nousresearch-meta-llama_4939_v49
developer_uid: end_to_end_test
alignment_samples: 2903
alignment_score: 1.4118862093687279
best_of: 4
celo_rating: 1182.23
display_name: nousresearch-meta-llama_4939_v49
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v49
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 2903
num_wins: 1223
propriety_score: 0.726457399103139
propriety_total_count: 223.0
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-30T23:50:04+00:00
us_pacific_date: 2024-08-30
win_ratio: 0.4212883224250775
Resubmit model
pipeline run %s
pipeline run stage %s
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4939-v49-mkmlizer
Waiting for job on nousresearch-meta-llama-4939-v49-mkmlizer to finish
nousresearch-meta-llama-4939-v49-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v49-mkmlizer: ║ _____ __ __ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ /___/ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ Version: 0.10.1 ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ https://mk1.ai ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ The license key for the current software has been verified as ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ belonging to: ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ Chai Research Corp. ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
nousresearch-meta-llama-4939-v49-mkmlizer: ║ ║
nousresearch-meta-llama-4939-v49-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v49-mkmlizer: Downloaded to shared memory in 33.832s
nousresearch-meta-llama-4939-v49-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprscri83p, device:0
nousresearch-meta-llama-4939-v49-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v49-mkmlizer: quantized model in 26.666s
nousresearch-meta-llama-4939-v49-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 60.498s
nousresearch-meta-llama-4939-v49-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v49-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v49-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v49
nousresearch-meta-llama-4939-v49-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v49/config.json
nousresearch-meta-llama-4939-v49-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v49/special_tokens_map.json
nousresearch-meta-llama-4939-v49-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v49/tokenizer_config.json
nousresearch-meta-llama-4939-v49-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v49/tokenizer.json
nousresearch-meta-llama-4939-v49-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v49/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v49-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s]Loading 0: 2%|▏ | 5/291 [00:00<00:08, 34.79it/s]Loading 0: 4%|▍ | 13/291 [00:00<00:04, 55.83it/s]Loading 0: 7%|▋ | 19/291 [00:00<00:05, 48.69it/s]Loading 0: 9%|▊ | 25/291 [00:00<00:05, 48.14it/s]Loading 0: 11%|█ | 31/291 [00:00<00:05, 51.36it/s]Loading 0: 13%|█▎ | 37/291 [00:00<00:05, 45.63it/s]Loading 0: 14%|█▍ | 42/291 [00:00<00:05, 44.59it/s]Loading 0: 17%|█▋ | 49/291 [00:01<00:04, 51.05it/s]Loading 0: 19%|█▉ | 55/291 [00:01<00:04, 48.33it/s]Loading 0: 21%|██ | 60/291 [00:01<00:05, 45.73it/s]Loading 0: 23%|██▎ | 67/291 [00:01<00:04, 49.94it/s]Loading 0: 25%|██▌ | 73/291 [00:01<00:04, 45.63it/s]Loading 0: 27%|██▋ | 78/291 [00:01<00:04, 44.28it/s]Loading 0: 29%|██▊ | 83/291 [00:01<00:06, 32.77it/s]Loading 0: 30%|██▉ | 87/291 [00:02<00:06, 32.57it/s]Loading 0: 32%|███▏ | 93/291 [00:02<00:05, 38.14it/s]Loading 0: 34%|███▎ | 98/291 [00:02<00:04, 40.06it/s]Loading 0: 36%|███▌ | 104/291 [00:02<00:04, 38.33it/s]Loading 0: 38%|███▊ | 112/291 [00:02<00:03, 47.37it/s]Loading 0: 41%|████ | 118/291 [00:02<00:04, 43.24it/s]Loading 0: 42%|████▏ | 123/291 [00:02<00:03, 42.83it/s]Loading 0: 45%|████▍ | 130/291 [00:02<00:03, 48.52it/s]Loading 0: 47%|████▋ | 136/291 [00:03<00:03, 45.46it/s]Loading 0: 48%|████▊ | 141/291 [00:03<00:03, 42.94it/s]Loading 0: 51%|█████ | 147/291 [00:03<00:03, 46.81it/s]Loading 0: 52%|█████▏ | 152/291 [00:03<00:02, 46.83it/s]Loading 0: 54%|█████▍ | 157/291 [00:03<00:02, 46.81it/s]Loading 0: 56%|█████▌ | 162/291 [00:03<00:02, 45.74it/s]Loading 0: 57%|█████▋ | 167/291 [00:03<00:03, 37.80it/s]Loading 0: 59%|█████▉ | 173/291 [00:03<00:02, 42.84it/s]Loading 0: 62%|██████▏ | 180/291 [00:04<00:02, 48.02it/s]Loading 0: 64%|██████▍ | 186/291 [00:04<00:02, 44.94it/s]Loading 0: 66%|██████▌ | 191/291 [00:04<00:03, 32.78it/s]Loading 0: 67%|██████▋ | 195/291 [00:04<00:03, 31.60it/s]Loading 0: 69%|██████▉ | 201/291 [00:04<00:02, 37.48it/s]Loading 0: 71%|███████ | 206/291 [00:04<00:02, 38.77it/s]Loading 0: 73%|███████▎ | 211/291 [00:04<00:02, 39.70it/s]Loading 0: 75%|███████▍ | 217/291 [00:05<00:01, 39.57it/s]Loading 0: 76%|███████▋ | 222/291 [00:05<00:01, 40.38it/s]Loading 0: 79%|███████▊ | 229/291 [00:05<00:01, 46.14it/s]Loading 0: 80%|████████ | 234/291 [00:05<00:01, 45.71it/s]Loading 0: 82%|████████▏ | 239/291 [00:05<00:01, 38.37it/s]Loading 0: 85%|████████▍ | 246/291 [00:05<00:01, 44.60it/s]Loading 0: 86%|████████▋ | 251/291 [00:05<00:00, 44.67it/s]Loading 0: 88%|████████▊ | 257/291 [00:06<00:00, 40.58it/s]Loading 0: 91%|█████████ | 265/291 [00:06<00:00, 47.84it/s]Loading 0: 93%|█████████▎| 271/291 [00:06<00:00, 44.56it/s]Loading 0: 95%|█████████▍| 276/291 [00:06<00:00, 43.72it/s]Loading 0: 97%|█████████▋| 281/291 [00:06<00:00, 44.89it/s]Loading 0: 98%|█████████▊| 286/291 [00:06<00:00, 39.07it/s]Loading 0: 100%|██████████| 291/291 [00:12<00:00, 3.11it/s]
Job nousresearch-meta-llama-4939-v49-mkmlizer completed after 87.39s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v49-mkmlizer
Pipeline stage MKMLizer completed in 88.29s
pipeline run stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.25s
pipeline run stage %s
Running pipeline stage MKMLDeployer
Creating inference service nousresearch-meta-llama-4939-v49
Waiting for inference service nousresearch-meta-llama-4939-v49 to be ready
Inference service nousresearch-meta-llama-4939-v49 ready after 191.71816301345825s
Pipeline stage MKMLDeployer completed in 192.55s
pipeline run stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3587560653686523s
Received healthy response to inference request in 1.4689640998840332s
Received healthy response to inference request in 1.855116844177246s
Received healthy response to inference request in 1.3969287872314453s
Received healthy response to inference request in 1.200087070465088s
5 requests
0 failed requests
5th percentile: 1.2394554138183593
10th percentile: 1.2788237571716308
20th percentile: 1.3575604438781739
30th percentile: 1.4113358497619628
40th percentile: 1.440149974822998
50th percentile: 1.4689640998840332
60th percentile: 1.6234251976013183
70th percentile: 1.7778862953186034
80th percentile: 1.9558446884155274
90th percentile: 2.1573003768920898
95th percentile: 2.258028221130371
99th percentile: 2.338610496520996
mean time: 1.655970573425293
Pipeline stage StressChecker completed in 10.55s
pipeline run stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.40s
nousresearch-meta-llama_4939_v49 status is now deployed due to DeploymentManager action
pipeline run %s
pipeline run stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.14s
pipeline run stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v49-profiler
Waiting for inference service nousresearch-meta-llama-4939-v49-profiler to be ready
Inference service nousresearch-meta-llama-4939-v49-profiler ready after 190.5164134502411s
Pipeline stage MKMLProfilerDeployer completed in 190.90s
pipeline run stage %s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.37s
pipeline run stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service nousresearch-meta-llama-4939-v49-profiler is running
Tearing down inference service nousresearch-meta-llama-4939-v49-profiler
Service nousresearch-meta-llama-4939-v49-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 2.02s
nousresearch-meta-llama_4939_v49 status is now inactive due to admin request
nousresearch-meta-llama_4939_v49 status is now torndown due to DeploymentManager action
