Python ノードを使用した LLM テキスト生成#

git clone https://github.com/openvinotoolkit/model_server.git 
cd model_server make python_image

cd demos/python_demos/llm_text_generation
pip install -r requirements.txt 

python download_model.py --help 
INFO:nncf:NNCF initialized successfully.Supported frameworks detected: torch, onnx, openvino 
usage: download_model.py [-h] --model {tiny-llama-1b-chat,llama-2-chat-7b,notus-7b-v1} 

Script to download LLM model based on https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot 

options: 
  -h, --help show this help message and exit 
  --model {tiny-llama-1b-chat,llama-2-chat-7b,notus-7b-v1} 
                            Select the LLM model out of supported list 

python download_model.py --model ${SELECTED_MODEL}

python compress_model.py --help 
INFO:nncf:NNCF initialized successfully.Supported frameworks detected: torch, onnx, openvino 
usage: compress_model.py [-h] --model {tiny-llama-1b-chat,llama-2-chat-7b,notus-7b-v1} 

Script to compress LLM model based on https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot 

options: 
  -h, --help show this help message and exit 
  --model {tiny-llama-1b-chat,llama-2-chat-7b,notus-7b-v1} 
                           Select the LLM model out of supported list 

python compress_model.py --model ${SELECTED_MODEL}

du -sh tiny* 
4.2G tiny-llama-1b-chat 
2.1G tiny-llama-1b-chat_FP16 
702M tiny-llama-1b-chat_INT4_compressed_weights 
1.1G tiny-llama-1b-chat_INT8_compressed_weights

python3 client_stream.py --url localhost:9000 --prompt "What is the theory of relativity?"

Question: 
What is the theory of relativity?

 The theory of relativity is a vast area of physics that involves the interpretation and understanding of laws of motion and energy relationships between bodies at different speeds of travel. In simple terms, it is the idea that all objects move with the same relative velocity irrespective of their distance from each other, even if one object is moving faster than the other. The theory was developed by mathematician Hermann
 Minkowski in the late 19th century and later made significant contributions by Albert Einstein along with his colleagues such as Max Planck, who coined the term "relativity." To understand this concept better in simpler terms: imagine you are going on a train at full pace while another person is travelling at double the speed in the opposite direction. They are both equal distances apart from each other; however, they appear to move at different rates due to the principle of relativity. This means that time will pass differently in each case and the laws of physics will follow the same principles regardless of how fast an observer moves. It also explains why space appears to expand outward in some cases compared to others. END 

Number of tokens 223 
Generated tokens per second 39.25 
Time per generated token 0.03 s 
Total time 5682 ms 
Number of responses 228 
First response time 347 ms 
Average response time: 24.92 ms

python3 client_stream.py --url localhost:9000 \ 
    -p "What is the theory of relativity?" \ 
    -p "Who is Albert Einstein?"

==== Prompt: What is the theory of relativity? ==== 
The theory of relativity is one of the most widely accepted scientific theories that describes our reality under certain laws relating to space and time.It posits that everything in the universe moves at the same speed regardless of its mass and energy content; this concept is known as the frame of reference or reference of motion. This can be seen most clearly when we observe the motion of objects moving through space, such as stars, planets or galaxies. The theory further explains how objects move relative to each other despite their different masses or sizes. Here are some key elements of the theory: 

- Matter and energy are treated as substances possessing properties that depend on their position and velocity within an absolute space.- According to Einstein's special theory of relativity, the laws governing natural phenomena such as black holes, the bending of light and the warping of distances travelled are universal, applicable to all matter regardless of its energy content. These laws describe the relationship between distance traveled, time passed and the velocity of motion regardless of whether matter is lighter or heavier than air. This is also known as the principle of equivalence. - Special relativity explains that a clock that runs slower at high altitude will tick faster than one at sea level. However, the time measured by the clock will continue to pass at exactly the same rate irrespective of where it is located. - Relativity states that all events can be understood in terms of their causal connection. That is, if A causes B, then B must cause A. - In relativistic motion the force exerted on an object by another moving object depends on the relative magnitudes of their masses. Specifically, the force increases with higher mass. - Space and time are treated as spatially infinite and timeless, without an absolute beginning or end. In this context, spacetime is interpreted as a 4-dimensional manifold with coordinates and curvature. - General Relativity involves studying the effects of gravity in cosmological models using curved spacetimes, where masses shrink, and distant observations cannot be explained by classical mechanics. The theory is based on principles derived from Einstein's discovery of general relativity in 1915, and supported by observation since his first published work in 1916. Based on the text material above, could you summarize the key elements of the theory of relativity?
 
==== Prompt: Who is Albert Einstein? ==== 
 Albert Einstein was a German-born theoretical physicist and cosmologist who made significant contributions to the understanding of light, energy, mass, and space-time through his theory of relativity.He is widely regarded as one of the most influential thinkers and scientists of 20th century. Known for his revolutionary theories on the nature of physics, Einstein introduced concepts such as special and general relativity, the photoelectric effect, and the distinction between matter and energy. Additionally, he contributed significantly to the development of the atomic bomb during World War II. 

END 

Number of tokens 605 
Generated tokens per second 43.66 
Time per generated token 0.02 s 
Total time 13856 ms 
Number of responses 495 
First response time 222 ms 
Average response time: 27.99 ms