About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / virtual / kvm / ppc-pv.txt


Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.

1	The PPC KVM paravirtual interface
2	=================================
3	
4	The basic execution principle by which KVM on PowerPC works is to run all kernel
5	space code in PR=1 which is user space. This way we trap all privileged
6	instructions and can emulate them accordingly.
7	
8	Unfortunately that is also the downfall. There are quite some privileged
9	instructions that needlessly return us to the hypervisor even though they
10	could be handled differently.
11	
12	This is what the PPC PV interface helps with. It takes privileged instructions
13	and transforms them into unprivileged ones with some help from the hypervisor.
14	This cuts down virtualization costs by about 50% on some of my benchmarks.
15	
16	The code for that interface can be found in arch/powerpc/kernel/kvm*
17	
18	Querying for existence
19	======================
20	
21	To find out if we're running on KVM or not, we leverage the device tree. When
22	Linux is running on KVM, a node /hypervisor exists. That node contains a
23	compatible property with the value "linux,kvm".
24	
25	Once you determined you're running under a PV capable KVM, you can now use
26	hypercalls as described below.
27	
28	KVM hypercalls
29	==============
30	
31	Inside the device tree's /hypervisor node there's a property called
32	'hypercall-instructions'. This property contains at most 4 opcodes that make
33	up the hypercall. To call a hypercall, just call these instructions.
34	
35	The parameters are as follows:
36	
37		Register	IN			OUT
38	
39		r0		-			volatile
40		r3		1st parameter		Return code
41		r4		2nd parameter		1st output value
42		r5		3rd parameter		2nd output value
43		r6		4th parameter		3rd output value
44		r7		5th parameter		4th output value
45		r8		6th parameter		5th output value
46		r9		7th parameter		6th output value
47		r10		8th parameter		7th output value
48		r11		hypercall number	8th output value
49		r12		-			volatile
50	
51	Hypercall definitions are shared in generic code, so the same hypercall numbers
52	apply for x86 and powerpc alike with the exception that each KVM hypercall
53	also needs to be ORed with the KVM vendor code which is (42 << 16).
54	
55	Return codes can be as follows:
56	
57		Code		Meaning
58	
59		0		Success
60		12		Hypercall not implemented
61		<0		Error
62	
63	The magic page
64	==============
65	
66	To enable communication between the hypervisor and guest there is a new shared
67	page that contains parts of supervisor visible register state. The guest can
68	map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE.
69	
70	With this hypercall issued the guest always gets the magic page mapped at the
71	desired location. The first parameter indicates the effective address when the
72	MMU is enabled. The second parameter indicates the address in real mode, if
73	applicable to the target. For now, we always map the page to -4096. This way we
74	can access it using absolute load and store functions. The following
75	instruction reads the first field of the magic page:
76	
77		ld	rX, -4096(0)
78	
79	The interface is designed to be extensible should there be need later to add
80	additional registers to the magic page. If you add fields to the magic page,
81	also define a new hypercall feature to indicate that the host can give you more
82	registers. Only if the host supports the additional features, make use of them.
83	
84	The magic page layout is described by struct kvm_vcpu_arch_shared
85	in arch/powerpc/include/asm/kvm_para.h.
86	
87	Magic page features
88	===================
89	
90	When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE,
91	a second return value is passed to the guest. This second return value contains
92	a bitmap of available features inside the magic page.
93	
94	The following enhancements to the magic page are currently available:
95	
96	  KVM_MAGIC_FEAT_SR		Maps SR registers r/w in the magic page
97	  KVM_MAGIC_FEAT_MAS0_TO_SPRG7	Maps MASn, ESR, PIR and high SPRGs
98	
99	For enhanced features in the magic page, please check for the existence of the
100	feature before using them!
101	
102	Magic page flags
103	================
104	
105	In addition to features that indicate whether a host is capable of a particular
106	feature we also have a channel for a guest to tell the guest whether it's capable
107	of something. This is what we call "flags".
108	
109	Flags are passed to the host in the low 12 bits of the Effective Address.
110	
111	The following flags are currently available for a guest to expose:
112	
113	  MAGIC_PAGE_FLAG_NOT_MAPPED_NX Guest handles NX bits correctly wrt magic page
114	
115	MSR bits
116	========
117	
118	The MSR contains bits that require hypervisor intervention and bits that do
119	not require direct hypervisor intervention because they only get interpreted
120	when entering the guest or don't have any impact on the hypervisor's behavior.
121	
122	The following bits are safe to be set inside the guest:
123	
124	  MSR_EE
125	  MSR_RI
126	
127	If any other bit changes in the MSR, please still use mtmsr(d).
128	
129	Patched instructions
130	====================
131	
132	The "ld" and "std" instructions are transformed to "lwz" and "stw" instructions
133	respectively on 32 bit systems with an added offset of 4 to accommodate for big
134	endianness.
135	
136	The following is a list of mapping the Linux kernel performs when running as
137	guest. Implementing any of those mappings is optional, as the instruction traps
138	also act on the shared page. So calling privileged instructions still works as
139	before.
140	
141	From			To
142	====			==
143	
144	mfmsr	rX		ld	rX, magic_page->msr
145	mfsprg	rX, 0		ld	rX, magic_page->sprg0
146	mfsprg	rX, 1		ld	rX, magic_page->sprg1
147	mfsprg	rX, 2		ld	rX, magic_page->sprg2
148	mfsprg	rX, 3		ld	rX, magic_page->sprg3
149	mfsrr0	rX		ld	rX, magic_page->srr0
150	mfsrr1	rX		ld	rX, magic_page->srr1
151	mfdar	rX		ld	rX, magic_page->dar
152	mfdsisr	rX		lwz	rX, magic_page->dsisr
153	
154	mtmsr	rX		std	rX, magic_page->msr
155	mtsprg	0, rX		std	rX, magic_page->sprg0
156	mtsprg	1, rX		std	rX, magic_page->sprg1
157	mtsprg	2, rX		std	rX, magic_page->sprg2
158	mtsprg	3, rX		std	rX, magic_page->sprg3
159	mtsrr0	rX		std	rX, magic_page->srr0
160	mtsrr1	rX		std	rX, magic_page->srr1
161	mtdar	rX		std	rX, magic_page->dar
162	mtdsisr	rX		stw	rX, magic_page->dsisr
163	
164	tlbsync			nop
165	
166	mtmsrd	rX, 0		b	<special mtmsr section>
167	mtmsr	rX		b	<special mtmsr section>
168	
169	mtmsrd	rX, 1		b	<special mtmsrd section>
170	
171	[Book3S only]
172	mtsrin	rX, rY		b	<special mtsrin section>
173	
174	[BookE only]
175	wrteei	[0|1]		b	<special wrteei section>
176	
177	
178	Some instructions require more logic to determine what's going on than a load
179	or store instruction can deliver. To enable patching of those, we keep some
180	RAM around where we can live translate instructions to. What happens is the
181	following:
182	
183		1) copy emulation code to memory
184		2) patch that code to fit the emulated instruction
185		3) patch that code to return to the original pc + 4
186		4) patch the original instruction to branch to the new code
187	
188	That way we can inject an arbitrary amount of code as replacement for a single
189	instruction. This allows us to check for pending interrupts when setting EE=1
190	for example.
191	
192	Hypercall ABIs in KVM on PowerPC
193	=================================
194	1) KVM hypercalls (ePAPR)
195	
196	These are ePAPR compliant hypercall implementation (mentioned above). Even
197	generic hypercalls are implemented here, like the ePAPR idle hcall. These are
198	available on all targets.
199	
200	2) PAPR hypercalls
201	
202	PAPR hypercalls are needed to run server PowerPC PAPR guests (-M pseries in QEMU).
203	These are the same hypercalls that pHyp, the POWER hypervisor implements. Some of
204	them are handled in the kernel, some are handled in user space. This is only
205	available on book3s_64.
206	
207	3) OSI hypercalls
208	
209	Mac-on-Linux is another user of KVM on PowerPC, which has its own hypercall (long
210	before KVM). This is supported to maintain compatibility. All these hypercalls get
211	forwarded to user space. This is only useful on book3s_32, but can be used with
212	book3s_64 as well.
Hide Line Numbers


About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog