delta/mlua
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

New approach for ref types, use an auxillary thread stack

Browse source 
Vastly simpler and less magical than using a fixed size magical section of the
active stack, and seems to be no slower.  The only real downside is that
it *seems* extremely extremely hacky (and to be fair, it is).
This commit is contained in:
kyren 2018-03-28 01:09:51 -04:00
parent 60bfe1fb07
commit 71f3dd50a1
10 changed files with 280 additions and 455 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
CHANGELOG.md
View file

@ -1,26 +1,26 @@
## [0.14.0]
- Lots of performance improvements, including one major change: Lua handles no
longer necessarily point into the registry, they now can instead point into a
pre-allocated stack area. All together, you can expect (VERY rough estimate)
somewhere on the order of 30%-60% CPU time reduction in the cost of bindings,
depending on usage patterns.
longer use the registry, they now instead are stored in an auxillary thread
stack created solely for the purpose of holding ref values. This may seem
extremely weird, but it is vastly faster than using the registry or raw tables
for this purpose. The first attempt here was to use the same stack and to do
a lot of complex bookkeeping to manage the references, and this DOES work, but
it comes with a lot of complexity and downsides along with it. The second
approach of using an auxillary thread turned out to be equally fast and with
no real downsides over the previous approach. With all the performance
improvements together, you can expect (VERY rough estimate) somewhere on the
order of 30%-60% CPU time reduction in the cost of bindings, depending on
usage patterns.
- Addition of some simple criterion.rs based benchmarking. This is the first
`rlua` release to focus on performance, but performance will hopefully remain
a focus going forward.
- API incompatible change: Lua handles now must ONLY be used with the `Lua`
instance from which they were derived. Before, this would work if both
instances were from the same base state, but are now restricted to ONLY the
matching `Lua` instance itself. This was previously only possible through
unsafe code, `Lua::scope`, and things like the `rental` crate which allow self
borrowing. For `Lua::scope` functions, you can use `Function::bind`, and for
everything else you can use the `RegistryKey` API.
- Another API incompatible change: `Lua` (and associated handle values) are no
longer `UnwindSafe` / `RefUnwindSafe`. They should not have been marked as
such before, because they are *extremely* internally mutable, so this can be
considered a bugfix. All `rlua` types should actually be perfectly panic safe
as far as *internal* invariants are concerned, but (afaict) they should not be
marked as `UnwindSafe` due to internal mutability and thus potentially
breaking *user* invariants.
- API incompatible change: `Lua` is no longer `RefUnwindSafe` and associated
handle values are no longer `UnwindSafe` or `RefUnwindSafe`. They should not
have been marked as such before, because they are *extremely* internally
mutable, so this can be considered a bugfix. All `rlua` types should actually
be perfectly panic safe as far as *internal* invariants are concerned, but
(afaict) they should not be marked as `RefUnwindSafe` due to internal
mutability and thus potentially breaking *user* invariants.
- Several Lua stack checking bugs have been fixed that could have lead to
unsafety in release mode.

5
README.md
View file

@ -64,9 +64,8 @@ considered a bug.
Caveats to the panic / abort guarantee:
* `rlua` reserves the right to panic on API usage errors. Currently, the only
time this will happen is when passed a handle type from a mismatched `Lua`
instance, or when accessing outer level `Lua` instances and handles inside
an inner callback.
time this will happen is when passed a handle type from a `Lua` instance
that does not share the same main state.
* Currently, there are no memory or execution limits on scripts, so untrusted
scripts can always at minimum infinite loop or allocate arbitrary amounts of
memory.

599
src/lua.rs
View file

@ -1,6 +1,6 @@
use std::{cmp, mem, ptr, str};
use std::{mem, ptr, str};
use std::sync::{Arc, Mutex};
use std::cell::{Cell, RefCell};
use std::cell::{RefCell, UnsafeCell};
use std::ffi::CString;
use std::any::TypeId;
use std::marker::PhantomData;
@ -12,11 +12,11 @@ use libc;
use ffi;
use error::{Error, Result};
use util::{assert_stack, callback_error, check_stack, gc_guard, get_userdata, get_wrapped_error,
init_error_metatables, pop_error, protect_lua, protect_lua_closure, push_string,
push_userdata, push_wrapped_error, safe_pcall, safe_xpcall, userdata_destructor,
StackGuard};
init_error_metatables, main_state, pop_error, protect_lua, protect_lua_closure,
push_string, push_userdata, push_wrapped_error, safe_pcall, safe_xpcall,
userdata_destructor, StackGuard};
use value::{FromLua, FromLuaMulti, MultiValue, Nil, ToLua, ToLuaMulti, Value};
use types::{Callback, Integer, LightUserData, LuaRef, Number, RefType, RegistryKey};
use types::{Callback, Integer, LightUserData, LuaRef, Number, RegistryKey};
use string::String;
use table::Table;
use function::Function;
@ -27,8 +27,10 @@ use scope::Scope;
/// Top level Lua struct which holds the Lua state itself.
pub struct Lua {
pub(crate) state: *mut ffi::lua_State,
recursion_level: usize,
ref_stack_slots: [Cell<usize>; REF_STACK_SIZE as usize],
main_state: *mut ffi::lua_State,
ephemeral: bool,
// Lua has lots of interior mutability, should not be RefUnwindSafe
_phantom: PhantomData<UnsafeCell<()>>,
}
unsafe impl Send for Lua {}
@ -36,31 +38,15 @@ unsafe impl Send for Lua {}
impl Drop for Lua {
fn drop(&mut self) {
unsafe {
if cfg!(debug_assertions) {
for use_count in &self.ref_stack_slots {
rlua_assert!(use_count.get() == 0, "live stack reference detected");
}
if self.recursion_level == 0 {
let top = ffi::lua_gettop(self.state);
rlua_assert!(
top == REF_STACK_SIZE,
"stack problem detected, stack top is {}",
top - REF_STACK_SIZE
);
}
}
if self.recursion_level == 0 {
let extra_data = *(ffi::lua_getextraspace(self.state) as *mut *mut ExtraData);
if !self.ephemeral {
let extra = extra_data(self.state);
rlua_debug_assert!(
(*extra_data).recursion_level == 0,
"Lua recursion level nonzero on Lua drop"
ffi::lua_gettop((*extra).ref_thread) == (*extra).ref_stack_max
&& (*extra).ref_stack_max as usize == (*extra).ref_free.len(),
"reference leak detected"
);
*(*extra_data).registry_unref_list.lock().unwrap() = None;
Box::from_raw(extra_data);
*(*extra).registry_unref_list.lock().unwrap() = None;
Box::from_raw(extra);
ffi::lua_close(self.state);
}
@ -71,7 +57,7 @@ impl Drop for Lua {
impl Lua {
/// Creates a new Lua state and loads standard library without the `debug` library.
pub fn new() -> Lua {
unsafe { Lua::create_lua(false) }
unsafe { create_lua(false) }
}
/// Creates a new Lua state and loads the standard library including the `debug` library.
@ -79,7 +65,7 @@ impl Lua {
/// The debug library is very unsound, loading it and using it breaks all the guarantees of
/// rlua.
pub unsafe fn new_with_debug() -> Lua {
Lua::create_lua(true)
create_lua(true)
}
/// Loads a chunk of Lua code and returns it as a function.
@ -546,7 +532,7 @@ impl Lua {
Ok(RegistryKey {
registry_id,
unref_list: (*ExtraData::get(self.state)).registry_unref_list.clone(),
unref_list: (*extra_data(self.state)).registry_unref_list.clone(),
})
}
}
@ -606,7 +592,7 @@ impl Lua {
unsafe {
Arc::ptr_eq(
&key.unref_list,
&(*ExtraData::get(self.state)).registry_unref_list,
&(*extra_data(self.state)).registry_unref_list,
)
}
}
@ -619,7 +605,7 @@ impl Lua {
pub fn expire_registry_values(&self) {
unsafe {
let unref_list = mem::replace(
&mut *(*ExtraData::get(self.state))
&mut *(*extra_data(self.state))
.registry_unref_list
.lock()
.unwrap(),
@ -735,139 +721,48 @@ impl Lua {
}
}
// Used 1 stack space, does not call checkstack
pub(crate) unsafe fn push_ref(&self, lref: &LuaRef) {
// Pushes a LuaRef value onto the stack, uses 1 stack space, does not call checkstack
pub(crate) unsafe fn push_ref<'lua>(&'lua self, lref: &LuaRef<'lua>) {
assert!(
self.is_active(),
"parent Lua instance accessed inside callback"
lref.lua.main_state == self.main_state,
"Lua instance passed Value created from a different main Lua state"
);
assert!(
lref.lua as *const Lua == self as *const Lua,
"Lua instance passed Value created from a different Lua"
);
match lref.ref_type {
RefType::Nil => ffi::lua_pushnil(self.state),
RefType::Stack { stack_slot } => {
ffi::lua_pushvalue(self.state, stack_slot);
}
RefType::Registry { registry_id } => {
ffi::lua_rawgeti(
self.state,
ffi::LUA_REGISTRYINDEX,
registry_id as ffi::lua_Integer,
);
}
}
let extra = extra_data(self.state);
ffi::lua_pushvalue((*extra).ref_thread, lref.index);
ffi::lua_xmove((*extra).ref_thread, self.state, 1);
}
// Pops the topmost element of the stack and stores a reference to it in the
// registry.
// Pops the topmost element of the stack and stores a reference to it. This pins the object,
// preventing garbage collection until the returned `LuaRef` is dropped.
//
// This pins the object, preventing garbage collection until the returned
// `LuaRef` is dropped.
//
// pop_ref uses 1 extra stack space and does not call checkstack
pub(crate) unsafe fn pop_ref(&self) -> LuaRef {
assert!(
self.is_active(),
"parent Lua instance accessed inside callback"
);
// References are stored in the stack of a specially created auxillary thread that exists only
// to store reference values. This is much faster than storing these in the registry, and also
// much more flexible and requires less bookkeeping than storing them directly in the currently
// used stack. The implementation is somewhat biased towards the use case of a relatively small
// number of short term references being created, and `RegistryKey` being used for long term
// references.
pub(crate) unsafe fn pop_ref<'lua>(&'lua self) -> LuaRef<'lua> {
let extra = extra_data(self.state);
ffi::lua_xmove(self.state, (*extra).ref_thread, 1);
let index = ref_stack_pop(extra);
LuaRef { lua: self, index }
}
for i in 0..REF_STACK_SIZE {
let ref_slot = &self.ref_stack_slots[i as usize];
if ref_slot.get() == 0 {
ref_slot.set(1);
ffi::lua_replace(self.state, i + 1);
return LuaRef {
lua: self,
ref_type: RefType::Stack { stack_slot: i + 1 },
};
}
}
let registry_id = gc_guard(self.state, || {
ffi::luaL_ref(self.state, ffi::LUA_REGISTRYINDEX)
});
if registry_id == ffi::LUA_REFNIL {
LuaRef {
lua: self,
ref_type: RefType::Nil,
}
} else {
LuaRef {
lua: self,
ref_type: RefType::Registry {
registry_id: registry_id,
},
}
pub(crate) fn clone_ref<'lua>(&'lua self, lref: &LuaRef<'lua>) -> LuaRef<'lua> {
unsafe {
let extra = extra_data(self.state);
ffi::lua_pushvalue((*extra).ref_thread, lref.index);
let index = ref_stack_pop(extra);
LuaRef { lua: self, index }
}
}
pub(crate) fn clone_ref(&self, lref: &LuaRef) -> LuaRef {
assert!(
self.is_active(),
"parent Lua instance accessed inside callback"
);
pub(crate) fn drop_ref<'lua>(&'lua self, lref: &mut LuaRef<'lua>) {
unsafe {
match lref.ref_type {
RefType::Nil => LuaRef {
lua: self,
ref_type: RefType::Nil,
},
RefType::Stack { stack_slot } => {
let ref_slot = &self.ref_stack_slots[(stack_slot - 1) as usize];
ref_slot.set(ref_slot.get() + 1);
LuaRef {
lua: self,
ref_type: RefType::Stack { stack_slot },
}
}
RefType::Registry { registry_id } => {
assert_stack(self.state, 2);
ffi::lua_rawgeti(
self.state,
ffi::LUA_REGISTRYINDEX,
registry_id as ffi::lua_Integer,
);
let registry_id = gc_guard(self.state, || {
ffi::luaL_ref(self.state, ffi::LUA_REGISTRYINDEX)
});
LuaRef {
lua: self,
ref_type: RefType::Registry {
registry_id: registry_id,
},
}
}
}
}
}
pub(crate) fn drop_ref(&self, lref: &mut LuaRef) {
assert!(
self.is_active(),
"parent Lua instance accessed inside callback"
);
unsafe {
match lref.ref_type {
RefType::Nil => {}
RefType::Stack { stack_slot } => {
let ref_slot = &self.ref_stack_slots[(stack_slot - 1) as usize];
let ref_count = ref_slot.get();
rlua_debug_assert!(ref_count > 0, "ref slot use count has gone below zero");
ref_slot.set(ref_count - 1);
if ref_count == 1 {
ffi::lua_pushnil(self.state);
ffi::lua_replace(self.state, stack_slot);
}
}
RefType::Registry { registry_id } => {
ffi::luaL_unref(self.state, ffi::LUA_REGISTRYINDEX, registry_id);
}
}
let extra = extra_data(self.state);
ffi::lua_pushnil((*extra).ref_thread);
ffi::lua_replace((*extra).ref_thread, lref.index);
(*extra).ref_free.push(lref.index);
}
}
@ -892,7 +787,7 @@ impl Lua {
}
}
if let Some(table_id) = (*ExtraData::get(self.state))
if let Some(table_id) = (*extra_data(self.state))
.registered_userdata
.get(&TypeId::of::<T>())
{
@ -996,7 +891,7 @@ impl Lua {
let id = gc_guard(self.state, || {
ffi::luaL_ref(self.state, ffi::LUA_REGISTRYINDEX)
});
(*ExtraData::get(self.state))
(*extra_data(self.state))
.registered_userdata
.insert(TypeId::of::<T>(), id);
Ok(id)
@ -1011,15 +906,25 @@ impl Lua {
if ffi::lua_type(state, ffi::lua_upvalueindex(1)) == ffi::LUA_TNIL {
return Err(Error::CallbackDestructed);
}
let recursion_guard = RecursionGuard::new(ExtraData::get(state));
let nargs = ffi::lua_gettop(state);
if nargs < ffi::LUA_MINSTACK {
check_stack(state, ffi::LUA_MINSTACK - nargs)?;
}
let lua = Lua {
state: state,
recursion_level: recursion_guard.recursion_level(),
ref_stack_slots: Default::default(),
main_state: main_state(state),
ephemeral: true,
_phantom: PhantomData,
};
let args = lua.setup_callback_stack()?;
let mut args = MultiValue::new();
args.reserve(nargs as usize);
for _ in 0..nargs {
args.push_front(lua.pop_value());
}
let func = get_userdata::<Callback>(state, ffi::lua_upvalueindex(1));
let results = (*func)(&lua, args)?;
@ -1075,249 +980,151 @@ impl Lua {
Ok(AnyUserData(self.pop_ref()))
}
unsafe fn create_lua(load_debug: bool) -> Lua {
unsafe extern "C" fn allocator(
_: *mut c_void,
ptr: *mut c_void,
_: usize,
nsize: usize,
) -> *mut c_void {
if nsize == 0 {
libc::free(ptr as *mut libc::c_void);
ptr::null_mut()
} else {
let p = libc::realloc(ptr as *mut libc::c_void, nsize);
if p.is_null() {
// We require that OOM results in an abort, and that the lua allocator function
// never errors. Since this is what rust itself normally does on OOM, this is
// not really a huge loss. Importantly, this allows us to turn off the gc, and
// then know that calling Lua API functions marked as 'm' will not result in a
// 'longjmp' error while the gc is off.
abort!("out of memory in Lua allocation, aborting!");
} else {
p as *mut c_void
}
}
}
let state = ffi::lua_newstate(allocator, ptr::null_mut());
// Ignores or `unwrap()`s 'm' errors, because this is assuming that nothing in the lua
// standard library will have a `__gc` metamethod error.
// Do not open the debug library, it can be used to cause unsafety.
ffi::luaL_requiref(state, cstr!("_G"), ffi::luaopen_base, 1);
ffi::luaL_requiref(state, cstr!("coroutine"), ffi::luaopen_coroutine, 1);
ffi::luaL_requiref(state, cstr!("table"), ffi::luaopen_table, 1);
ffi::luaL_requiref(state, cstr!("io"), ffi::luaopen_io, 1);
ffi::luaL_requiref(state, cstr!("os"), ffi::luaopen_os, 1);
ffi::luaL_requiref(state, cstr!("string"), ffi::luaopen_string, 1);
ffi::luaL_requiref(state, cstr!("utf8"), ffi::luaopen_utf8, 1);
ffi::luaL_requiref(state, cstr!("math"), ffi::luaopen_math, 1);
ffi::luaL_requiref(state, cstr!("package"), ffi::luaopen_package, 1);
ffi::lua_pop(state, 9);
init_error_metatables(state);
if load_debug {
ffi::luaL_requiref(state, cstr!("debug"), ffi::luaopen_debug, 1);
ffi::lua_pop(state, 1);
}
// Create the function metatable
ffi::lua_pushlightuserdata(
state,
&FUNCTION_METATABLE_REGISTRY_KEY as *const u8 as *mut c_void,
);
ffi::lua_newtable(state);
push_string(state, "__gc").unwrap();
ffi::lua_pushcfunction(state, userdata_destructor::<Callback>);
ffi::lua_rawset(state, -3);
push_string(state, "__metatable").unwrap();
ffi::lua_pushboolean(state, 0);
ffi::lua_rawset(state, -3);
ffi::lua_rawset(state, ffi::LUA_REGISTRYINDEX);
// Override pcall and xpcall with versions that cannot be used to catch rust panics.
ffi::lua_rawgeti(state, ffi::LUA_REGISTRYINDEX, ffi::LUA_RIDX_GLOBALS);
push_string(state, "pcall").unwrap();
ffi::lua_pushcfunction(state, safe_pcall);
ffi::lua_rawset(state, -3);
push_string(state, "xpcall").unwrap();
ffi::lua_pushcfunction(state, safe_xpcall);
ffi::lua_rawset(state, -3);
ffi::lua_pop(state, 1);
// Create ExtraData, and place it in the lua_State "extra space"
let extra_data = Box::into_raw(Box::new(ExtraData {
recursion_level: 0,
registered_userdata: HashMap::new(),
registry_unref_list: Arc::new(Mutex::new(Some(Vec::new()))),
}));
*(ffi::lua_getextraspace(state) as *mut *mut ExtraData) = extra_data;
rlua_debug_assert!(ffi::lua_gettop(state) == 0, "stack leak during creation");
assert_stack(state, REF_STACK_SIZE);
ffi::lua_settop(state, REF_STACK_SIZE);
Lua {
state,
recursion_level: 0,
ref_stack_slots: Default::default(),
}
}
// Set up the stack slot area in a callback, returning all arguments on the stack as a
// MultiValue. Also ensures that at least LUA_MINSTACK extra stack slots are available for use
// in the callback.
fn setup_callback_stack<'lua>(&'lua self) -> Result<MultiValue<'lua>> {
unsafe {
assert_stack(self.state, 2);
let nargs = ffi::lua_gettop(self.state);
let stack_nargs = cmp::min(REF_STACK_SIZE, nargs);
let mut args = MultiValue::new();
args.reserve(stack_nargs as usize);
// Convert all of the reference types in the ref stack area into LuaRef types in-place.
for i in 0..stack_nargs {
let n = stack_nargs - i;
let make_ref = || {
self.ref_stack_slots[(n - 1) as usize].set(1);
LuaRef {
lua: self,
ref_type: RefType::Stack { stack_slot: n },
}
};
match ffi::lua_type(self.state, n) {
ffi::LUA_TNIL => {
args.push_front(Value::Nil);
}
ffi::LUA_TBOOLEAN => {
args.push_front(Value::Boolean(ffi::lua_toboolean(self.state, n) != 0));
}
ffi::LUA_TLIGHTUSERDATA => {
args.push_front(Value::LightUserData(LightUserData(
ffi::lua_touserdata(self.state, n),
)));
}
ffi::LUA_TNUMBER => if ffi::lua_isinteger(self.state, n) != 0 {
args.push_front(Value::Integer(ffi::lua_tointeger(self.state, n)));
} else {
args.push_front(Value::Number(ffi::lua_tonumber(self.state, n)));
},
ffi::LUA_TSTRING => {
args.push_front(Value::String(String(make_ref())));
}
ffi::LUA_TTABLE => {
args.push_front(Value::Table(Table(make_ref())));
}
ffi::LUA_TFUNCTION => {
args.push_front(Value::Function(Function(make_ref())));
}
ffi::LUA_TUSERDATA => {
if let Some(err) = get_wrapped_error(self.state, n).as_ref() {
args.push_front(Value::Error(err.clone()));
} else {
args.push_front(Value::UserData(AnyUserData(make_ref())));
}
}
ffi::LUA_TTHREAD => {
args.push_front(Value::Thread(Thread(make_ref())));
}
_ => rlua_panic!("LUA_TNONE in setup_callback_stack"),
}
}
if nargs < REF_STACK_SIZE {
check_stack(self.state, REF_STACK_SIZE - nargs + ffi::LUA_MINSTACK)?;
ffi::lua_settop(self.state, REF_STACK_SIZE);
Ok(args)
} else if nargs > REF_STACK_SIZE {
if nargs - REF_STACK_SIZE < ffi::LUA_MINSTACK {
check_stack(self.state, ffi::LUA_MINSTACK - (nargs - REF_STACK_SIZE))?;
}
// If the total number of arguments exceeds the ref stack area, pop off the rest of
// the arguments as normal.
let mut extra_args = Vec::new();
extra_args.reserve((nargs - REF_STACK_SIZE) as usize);
for _ in REF_STACK_SIZE..nargs {
extra_args.push(self.pop_value());
}
extra_args.extend(args.into_vec_rev());
Ok(MultiValue::from_vec_rev(extra_args))
} else {
check_stack(self.state, ffi::LUA_MINSTACK)?;
Ok(args)
}
}
}
// Returns true if this is the "top" Lua instance. If this returns false, we are a level deeper
// into a callback and `ref_stack_slots` points to an area of the stack that is not currently
// accessible.
fn is_active(&self) -> bool {
unsafe { (*ExtraData::get(self.state)).recursion_level == self.recursion_level }
}
}
// Data associated with the main lua_State via lua_getextraspace.
struct ExtraData {
recursion_level: usize,
registered_userdata: HashMap<TypeId, c_int>,
registry_unref_list: Arc<Mutex<Option<Vec<c_int>>>>,
ref_thread: *mut ffi::lua_State,
ref_stack_size: c_int,
ref_stack_max: c_int,
ref_free: Vec<c_int>,
}
impl ExtraData {
unsafe fn get(state: *mut ffi::lua_State) -> *mut ExtraData {
*(ffi::lua_getextraspace(state) as *mut *mut ExtraData)
}
unsafe fn extra_data(state: *mut ffi::lua_State) -> *mut ExtraData {
*(ffi::lua_getextraspace(state) as *mut *mut ExtraData)
}
struct RecursionGuard(*mut ExtraData);
impl RecursionGuard {
unsafe fn new(ed: *mut ExtraData) -> RecursionGuard {
(*ed).recursion_level += 1;
RecursionGuard(ed)
}
fn recursion_level(&self) -> usize {
unsafe { (*self.0).recursion_level }
}
}
impl Drop for RecursionGuard {
fn drop(&mut self) {
unsafe {
(*self.0).recursion_level -= 1;
unsafe fn create_lua(load_debug: bool) -> Lua {
unsafe extern "C" fn allocator(
_: *mut c_void,
ptr: *mut c_void,
_: usize,
nsize: usize,
) -> *mut c_void {
if nsize == 0 {
libc::free(ptr as *mut libc::c_void);
ptr::null_mut()
} else {
let p = libc::realloc(ptr as *mut libc::c_void, nsize);
if p.is_null() {
// We require that OOM results in an abort, and that the lua allocator function
// never errors. Since this is what rust itself normally does on OOM, this is
// not really a huge loss. Importantly, this allows us to turn off the gc, and
// then know that calling Lua API functions marked as 'm' will not result in a
// 'longjmp' error while the gc is off.
abort!("out of memory in Lua allocation, aborting!");
} else {
p as *mut c_void
}
}
}
let state = ffi::lua_newstate(allocator, ptr::null_mut());
// Ignores or `unwrap()`s 'm' errors, because we are making the assumption that nothing in
// the lua standard library will have a `__gc` metamethod error.
// Do not open the debug library, it can be used to cause unsafety.
ffi::luaL_requiref(state, cstr!("_G"), ffi::luaopen_base, 1);
ffi::luaL_requiref(state, cstr!("coroutine"), ffi::luaopen_coroutine, 1);
ffi::luaL_requiref(state, cstr!("table"), ffi::luaopen_table, 1);
ffi::luaL_requiref(state, cstr!("io"), ffi::luaopen_io, 1);
ffi::luaL_requiref(state, cstr!("os"), ffi::luaopen_os, 1);
ffi::luaL_requiref(state, cstr!("string"), ffi::luaopen_string, 1);
ffi::luaL_requiref(state, cstr!("utf8"), ffi::luaopen_utf8, 1);
ffi::luaL_requiref(state, cstr!("math"), ffi::luaopen_math, 1);
ffi::luaL_requiref(state, cstr!("package"), ffi::luaopen_package, 1);
ffi::lua_pop(state, 9);
init_error_metatables(state);
if load_debug {
ffi::luaL_requiref(state, cstr!("debug"), ffi::luaopen_debug, 1);
ffi::lua_pop(state, 1);
}
// Create the function metatable
ffi::lua_pushlightuserdata(
state,
&FUNCTION_METATABLE_REGISTRY_KEY as *const u8 as *mut c_void,
);
ffi::lua_newtable(state);
push_string(state, "__gc").unwrap();
ffi::lua_pushcfunction(state, userdata_destructor::<Callback>);
ffi::lua_rawset(state, -3);
push_string(state, "__metatable").unwrap();
ffi::lua_pushboolean(state, 0);
ffi::lua_rawset(state, -3);
ffi::lua_rawset(state, ffi::LUA_REGISTRYINDEX);
// Override pcall and xpcall with versions that cannot be used to catch rust panics.
ffi::lua_rawgeti(state, ffi::LUA_REGISTRYINDEX, ffi::LUA_RIDX_GLOBALS);
push_string(state, "pcall").unwrap();
ffi::lua_pushcfunction(state, safe_pcall);
ffi::lua_rawset(state, -3);
push_string(state, "xpcall").unwrap();
ffi::lua_pushcfunction(state, safe_xpcall);
ffi::lua_rawset(state, -3);
ffi::lua_pop(state, 1);
// Create ref stack thread and place it in the registry to prevent it from being garbage
// collected.
let ref_thread = ffi::lua_newthread(state);
ffi::luaL_ref(state, ffi::LUA_REGISTRYINDEX);
// Create ExtraData, and place it in the lua_State "extra space"
let extra = Box::into_raw(Box::new(ExtraData {
registered_userdata: HashMap::new(),
registry_unref_list: Arc::new(Mutex::new(Some(Vec::new()))),
ref_thread,
// We need 1 extra stack space to move values in and out of the ref stack.
ref_stack_size: ffi::LUA_MINSTACK - 1,
ref_stack_max: 0,
ref_free: Vec::new(),
}));
*(ffi::lua_getextraspace(state) as *mut *mut ExtraData) = extra;
rlua_debug_assert!(ffi::lua_gettop(state) == 0, "stack leak during creation");
assert_stack(state, ffi::LUA_MINSTACK);
Lua {
state,
main_state: state,
ephemeral: false,
_phantom: PhantomData,
}
}
unsafe fn ref_stack_pop(extra: *mut ExtraData) -> c_int {
if let Some(free) = (*extra).ref_free.pop() {
ffi::lua_replace((*extra).ref_thread, free);
free
} else {
if (*extra).ref_stack_max >= (*extra).ref_stack_size {
// It is a user error to create enough references to exhaust the Lua max stack size for
// the ref thread.
if ffi::lua_checkstack((*extra).ref_thread, (*extra).ref_stack_size) == 0 {
panic!("cannot create a Lua reference, out of auxillary stack space");
}
(*extra).ref_stack_size *= 2;
}
(*extra).ref_stack_max += 1;
(*extra).ref_stack_max
}
}
const REF_STACK_SIZE: c_int = 16;
static FUNCTION_METATABLE_REGISTRY_KEY: u8 = 0;

17
src/table.rs
View file

@ -4,8 +4,8 @@ use std::os::raw::c_int;
use ffi;
use error::Result;
use util::{assert_stack, protect_lua, protect_lua_closure, StackGuard};
use types::{Integer, LuaRef, RefType};
use value::{FromLua, ToLua};
use types::{Integer, LuaRef};
use value::{FromLua, Nil, ToLua, Value};
/// Handle to an internal Lua table.
#[derive(Clone, Debug)]
@ -282,14 +282,9 @@ impl<'lua> Table<'lua> {
/// [`Result`]: type.Result.html
/// [Lua manual]: http://www.lua.org/manual/5.3/manual.html#pdf-next
pub fn pairs<K: FromLua<'lua>, V: FromLua<'lua>>(self) -> TablePairs<'lua, K, V> {
let next_key = Some(LuaRef {
lua: self.0.lua,
ref_type: RefType::Nil,
});
TablePairs {
table: self.0,
next_key,
next_key: Some(Nil),
_phantom: PhantomData,
}
}
@ -348,7 +343,7 @@ impl<'lua> Table<'lua> {
/// [`Table::pairs`]: struct.Table.html#method.pairs
pub struct TablePairs<'lua, K, V> {
table: LuaRef<'lua>,
next_key: Option<LuaRef<'lua>>,
next_key: Option<Value<'lua>>,
_phantom: PhantomData<(K, V)>,
}
@ -369,7 +364,7 @@ where
assert_stack(lua.state, 6);
lua.push_ref(&self.table);
lua.push_ref(&next_key);
lua.push_value(next_key);
if protect_lua_closure(lua.state, 2, ffi::LUA_MULTRET, |state| {
ffi::lua_next(state, -2) != 0
@ -377,7 +372,7 @@ where
ffi::lua_pushvalue(lua.state, -2);
let key = lua.pop_value();
let value = lua.pop_value();
self.next_key = Some(lua.pop_ref());
self.next_key = Some(lua.pop_value());
Some((key, value))
} else {

2
src/tests/mod.rs
View file

@ -687,7 +687,6 @@ fn scope_capture() {
}
#[test]
#[should_panic]
fn outer_lua_access() {
let lua = Lua::new();
let table = lua.create_table().unwrap();
@ -701,6 +700,7 @@ fn outer_lua_access() {
.call::<_, ()>(())
.unwrap();
});
assert_eq!(table.get::<_, String>("a").unwrap(), "b");
}
#[test]

35
src/types.rs
View file

@ -22,26 +22,34 @@ pub(crate) type Callback<'lua, 'a> =
/// An auto generated key into the Lua registry.
///
/// This is a handle to a value stored inside the Lua registry. It is not directly usable like the
/// `Table` or `Function` handle types, but it is much more flexible and can be used in many
/// situations where it is impossible to directly store a normal handle type. It is Send + Sync +
/// 'static, and can be used by *any* `Lua` instance as long as it is derived from the same
/// underlying main state (such as one received in a Rust callback). It is not automatically
/// garbage collected on Drop, but it can be removed with [`Lua::remove_registry_value`], and
/// instances not manually removed can be garbage collected with [`Lua::expire_registry_values`].
/// `Table` or `Function` handle types, but since it doesn't hold a reference to a parent Lua and is
/// Send + Sync + 'static, it is much more flexible and can be used in many situations where it is
/// impossible to directly store a normal handle type. It is not automatically garbage collected on
/// Drop, but it can be removed with [`Lua::remove_registry_value`], and instances not manually
/// removed can be garbage collected with [`Lua::expire_registry_values`].
///
/// Be warned, If you place this into Lua via a `UserData` type or a rust callback, it is *very
/// easy* to accidentally cause reference cycles that the Lua garbage collector cannot resolve.
/// Instead of placing a `RegistryKey` into a `UserData` type, prefer instead to use
/// `UserData::set_user_value` / `UserData::get_user_value`, and instead of moving a RegistryKey
/// into a callback, prefer `Lua::scope`.
/// [`UserData::set_user_value`] / [`UserData::get_user_value`], and instead of moving a RegistryKey
/// into a callback, prefer [`Lua::scope`].
///
/// [`Lua::remove_registry_value`]: struct.Lua.html#method.remove_registry_value
/// [`Lua::expire_registry_values`]: struct.Lua.html#method.expire_registry_values
/// [`Lua::scope`]: struct.Lua.html#method.scope
/// [`UserData::set_user_value`]: struct.UserData.html#method.set_user_value
/// [`UserData::get_user_value`]: struct.UserData.html#method.get_user_value
pub struct RegistryKey {
pub(crate) registry_id: c_int,
pub(crate) unref_list: Arc<Mutex<Option<Vec<c_int>>>>,
}
impl fmt::Debug for RegistryKey {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "RegistryKey({})", self.registry_id)
}
}
impl Drop for RegistryKey {
fn drop(&mut self) {
if let Some(list) = self.unref_list.lock().unwrap().as_mut() {
@ -62,21 +70,14 @@ impl RegistryKey {
}
}
#[derive(Debug)]
pub(crate) enum RefType {
Nil,
Stack { stack_slot: c_int },
Registry { registry_id: c_int },
}
pub(crate) struct LuaRef<'lua> {
pub(crate) lua: &'lua Lua,
pub(crate) ref_type: RefType,
pub(crate) index: c_int,
}
impl<'lua> fmt::Debug for LuaRef<'lua> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{:?}", self.ref_type)
write!(f, "Ref({})", self.index)
}
}

10
src/util.rs
View file

@ -9,7 +9,7 @@ use ffi;
use error::{Error, Result};
// Checks that Lua has enough free stack space for future stack operations. On failure, this will
// panic.
// panic with an internal error message.
pub unsafe fn assert_stack(state: *mut ffi::lua_State, amount: c_int) {
// TODO: This should only be triggered when there is a logic error in `rlua`. In the future,
// when there is a way to be confident about stack safety and test it, this could be enabled
@ -400,6 +400,14 @@ pub unsafe extern "C" fn safe_xpcall(state: *mut ffi::lua_State) -> c_int {
}
}
// Does not call lua_checkstack, uses 1 stack space.
pub unsafe fn main_state(state: *mut ffi::lua_State) -> *mut ffi::lua_State {
ffi::lua_rawgeti(state, ffi::LUA_REGISTRYINDEX, ffi::LUA_RIDX_MAINTHREAD);
let main_state = ffi::lua_tothread(state, -1);
ffi::lua_pop(state, 1);
main_state
}
// Pushes a WrappedError::Error to the top of the stack. Uses two stack spaces and does not call
// lua_checkstack.
pub unsafe fn push_wrapped_error(state: *mut ffi::lua_State, err: Error) {

8
src/value.rs
View file

@ -122,14 +122,6 @@ impl<'lua> MultiValue<'lua> {
v
}
pub(crate) fn from_vec_rev(v: Vec<Value<'lua>>) -> MultiValue<'lua> {
MultiValue(v)
}
pub(crate) fn into_vec_rev(self) -> Vec<Value<'lua>> {
self.0
}
pub(crate) fn reserve(&mut self, size: usize) {
self.0.reserve(size);
}

11
tests/compile-fail/not_refunwindsafe.rs Normal file
View file

@ -0,0 +1,11 @@
extern crate rlua;
use std::panic::catch_unwind;
use rlua::*;
fn main() {
let lua = Lua::new();
let _ = catch_unwind(|| lua.create_table().unwrap());
//~^ error: the trait bound `std::cell::UnsafeCell<()>: std::panic::RefUnwindSafe` is not satisfied in `rlua::Lua`
}

12
tests/compile-fail/ref_not_unwindsafe.rs Normal file
View file

@ -0,0 +1,12 @@
extern crate rlua;
use std::panic::catch_unwind;
use rlua::*;
fn main() {
let lua = Lua::new();
let table = lua.create_table().unwrap();
let _ = catch_unwind(move || table.set("a", "b").unwrap());
//~^ error: the trait bound `std::cell::UnsafeCell<()>: std::panic::RefUnwindSafe` is not satisfied in `rlua::Lua`
}